Tech Lead (Web Scraping & Data Pipelines) / Remote position

On behalf of our international client we are looking for a hands-on Team Leader to lead the Core Support team, a group of highly skilled 8-13 engineers responsible for maintaining, debugging, and optimizing our web scraping scripts and data pipeline infrastructure.

Key Responsibilities:

• Lead & Manage: Oversee the Core Support team, providing mentorship, setting priorities, and ensuring timely resolution of issues.
• Troubleshoot & Debug: Diagnose and Resolve Issues in scraping scripts, data pipelines, and internal infrastructure.
• Perform Website Analysis: Analyze Website Structures to design efficient and maintainable data collection scripts.
• Optimize & Enhance Scraping Scripts: Improve existing Web Scraping Solutions in Python to increase efficiency and reliability.
• Monitor & Maintain Data Pipelines: Maintain internal ETL processes. Troubleshoot data flow issues, and implement fixes across different pipeline stages.
• Ensure High Data Quality: Work closely with business teams to ensure extracted data meets internal guidelines and quality standards.
• Act as a Technical Authority: Provide expert guidance on anti-bot mechanisms, network debugging, and web scraping strategies.

Skills & Requirements:

• Strong Technical Leadership: Proven experience leading technical teams, especially in web scraping, data engineering, or system debugging.
• Web Scraping & Anti-Bot Expertise: Strong knowledge of Python and relevant libraries (Scrapy, Splash, Puppeteer, Requests, JSON, lxml, re, Pandas). Deep understanding of web scraping techniques, browser automation, and anti-bot countermeasures. Experience with headless browsers, user-agent spoofing, CAPTCHA solving, and rotating proxies.
• Data Pipeline & Debugging Skills: Understanding of ETL pipelines, data normalization, and data processing workflows. Strong familiarity with HTTP methods, web networking, API testing (Postman), and debugging tools (browser dev tools, proxies, logs analysis, etc.). Experience working with Linux command line for debugging and troubleshooting. Knowledge of SQL/NoSQL databases and experience handling large-scale data.
• Coding & Development Knowledge: Solid Object-Oriented Programming (OOP) principles. Proficient in Git/Bitbucket for version control and collaboration. Basic knowledge of HTML/CSS/JavaScript, with the ability to create a simple HTML page.
• Communication & Organizational Skills: Ability to communicate clearly in both written and spoken English. Experience working with internal stakeholders and handling technical support requests. Strong documentation skills for writing internal guidelines, debugging processes, and best practices.
• Optional: Degree in Computer Science, Information Systems, or a related field. Working experience with JavaScript. Familiarity with data pipeline architectures and technologies. Experience with cloud computing platforms (e.g., AWS, Azure, GCP).

What will you get:

• Work Model & Hours: Remote work opportunities and flexible working hours.
• Competitive Salary: up to 9,000 EUR before taxes / monthly (dependent on the applicant's experience and qualifications).
• Opportunity to lead a diverse talented team of engineers, analysts, and business professionals.
• Tackle complex challenges ensuring high data quality and system reliability.
• International work environment.
• Other benefits: health insurance, paid time off, professional development opportunities, etc.

If this job opportunity sounds exciting to you, we'd love to hear from you!

Please apply by sending your CV in English to e-mail: cv@darbaguru.lv