Job Department: Development
Job Type: Full Time
Job Location: Coimbatore
Exp. Range: 2-7 Years
Job Description:
We are seeking a highly skilled Web Scraping Engineer to join our team. You will be responsible for developing and maintaining web scraping scripts and tools, analyzing difficult sources, and defining the workflow and data processing. You will work closely with our data team to extract, clean, and organize data from various websites and APIs to support our business needs.
Requirement:
- Minimum of 2+ years of experience working with crawling systems/frameworks.
- Design and implement efficient web scraping scripts using Python
- Proficiency in Scrapy, Beautiful Soup, Selenium, Captcha solving technologies, Scraping data from anti-bot sites, Headless browsers, and Puppeteer.
- Bypass anti-bot protections (e.g., CAPTCHA, rate limiting, JavaScript rendering) responsibly and legally.
- Work closely with the Product Manager, Project Manager, and other project stakeholders to understand requirements.
- Solid understanding of Python skills and scraping libraries.
- Solid understanding of GIT.
- Solid understanding of proxies and proxy management.
- Knowledge of SQL and experience with relational databases(e.g., PostgreSQL, MongoDB)
Qualification Criteria and Responsibilities:
- Any degree in Computer Science, Engineering, or a similar relevant field.
- Enhance the existing data extraction/web scraping tools and workflow.
- Handle anti-bot measures (CAPTCHAs, proxies, headers)
- Identify and fix scraping-related issues.
- Scale the scrapers to download huge volumes of data.