DevOps Engineer | SDE - 1 | SDE - 2
Role: AI QA Automation Lead
Function: Quality Assurance
Location: Navi Mumbai, Maharashtra
Type: Full-time
Compensation: Competitive compensation package with equity participation
Industry: Artificial Intelligence, Technology
About Company
A groundbreaking AI initiative building the intelligence layer for India at massive scale. Backed by partnerships with global tech leaders like Meta and Google.
The company creates AI solutions designed for real adoption across languages, contexts, and daily needs. Engineered from day one for 100M+ users early with 1B-ready constraints on latency, cost, reliability, and safety.
Position Overview
You'll lead the quality strategy and automation frameworks for cutting-edge AI systems serving millions of users across India. You'll own and implement automation-first frameworks that ensure reliability and performance of AI platforms powering agentic workflows and LLM-based applications. This role puts you at the forefront of defining AI testing practices at unprecedented scale.
Role & Responsibilities
- Define and execute comprehensive quality strategy for LLM-based applications and AI systems
- Design and implement automation-first testing frameworks using AI-specific testing tools
- Lead QA automation initiatives for agentic workflows and conversational AI systems
- Build and maintain CI/CD pipelines with AI model validation and automated testing systems
- Mentor and guide QA engineering teams on AI testing frameworks and methodologies
- Collaborate with ML engineers to test model performance, accuracy, and reliability metrics
- Establish quality metrics and monitoring systems for production AI applications
Must Have Criteria
- 10-20 years of experience in QA automation for large-scale distributed platforms
- Hands-on experience with AI testing frameworks, Python automation using PyTest, and JavaScript automation using Playwright
- Expert-level proficiency in Python test automation with PyTest framework for AI systems
- Strong experience with JavaScript automation using Playwright for web-based AI applications
- Proven leadership experience managing QA automation teams of 5+ engineers
- Deep understanding of AI model testing, validation metrics, and performance benchmarking
- Experience implementing CI/CD pipelines with automated AI model evaluation and validation
Nice to Have
- Experience with LLM evaluation frameworks and custom AI testing metrics
- Background in testing machine learning pipelines and model deployment systems
- Experience with A/B testing frameworks for AI model performance comparison
- Knowledge of performance testing for AI inference systems and model serving
- Open source contributions to AI testing frameworks or ML evaluation tools
What We Offer
- Opportunity to shape AI testing practices at unprecedented scale
- Work with cutting-edge AI technologies and global tech partnerships
- Competitive compensation package with equity participation
- Access to world-class infrastructure and green energy powered systems
- Career growth in India's most ambitious AI initiative
FAQs
Are there any additional costs for payroll processing in multiple countries?
Throughout history, These artists have inspired countless others to explore the instrument and its diverse musical possibilities.
Mastering the Accordian rying ability.
The history of t, the Accordian has evolved, with various types emerging, including the piano accordian and the button accordian, each offering unique playing styles and sounds.
The Accordian is a versatile musical instrument that has been used in various genres, from folk to classical music. Its uniqu.
