Automation Tester-Standard Insurance

Must have skills: SDET skills, Azure devops, python, pytest test automation and Framework, AI ML practices, RAGAS, LLMs Brief JD: • Experience in designing LLM/RAG test automation solutions • Experience in testing for bias, drift, and fairness. • Familiarity with performance metrics (precision, recall, F1, ROC-AUC). • Knowledge of MLflow MLOps framework • Knowledge of tools for synthetic data generation and boundary testing • Knowledge of Azure DevOps for CI/CD pipeline development • Experience in RAG/pipeline evaluation frameworks - • pytest: Test automation framework • DeepEval or TruLens: LLM test assertions • RAGAS: RAG-specific metrics • Eleuther: LLM evaluation harness • Garak or Promptfoo: LLM red-teaming • Evidently: Drift/performance monitoring • Exposure to explainability frameworks (SHAP, LIME, Captum) • Handson experience in API and Database testing. • Practical knowledge of Databricks, Azure Cloud services land distributed data validation. • Proficiency with Azure DevOps pipelines YAML templates, agent pools, CICD workflows. • Experience in implementing AIML practices in testing e.g., test generation, anomaly detection, log analysis to improve test efficiency and coverage. • Familiarity with Cucumber (BDD) and test reporting frameworks (e.g., Allure). • Strong understanding of integration testing across Databricks streaming jobs, applications and microservices. • Experience with performance testing tools (JMeter, LoadRunner, or equivalent). • Experience with Pester framework for validating PowerShell scripts and infrastructure automation. • Solid understanding of Agile/Scrum methodology and handson usage of JIRA. • Excellent problem-solving, debugging, and communication skills, with the ability to advise development teams on testing best practices. Apply tot his job Apply To this Job