I'm in need of a robust data scraping script that can extract detailed information from a variety of job sites.
Key requirements:
- The tool must be able to extract job post data from 8 different job sites (e.g. [login to view URL], Jora, Adzuna etc) and compile it in a CSV format.
- The extracted information should include: Job titles, Company names, Locations, Salaries, Job Types (Full Time, Part Time, Contact, Casual), Occupation, Post date and application closing date, Job Description, Apply link, Images, Eligibility (if available), Other jobs posted by the same employer (or similar job listing), Job Reference or Job ID.
- Scrape jobs on a daily basis
- Exclude duplicate/multiple posting using Job Reference No. or Job ID.
- Save the scraped data on an AWS account
Skillset and experience required:
- Proficiency in web scraping and data extraction.
- Knowledge of job site structures and data extraction from them.
- Ability to compile and organize the extracted information in a readable and logical manner.
