Machine Learning Task Auditor

Role Overview

Help strengthen the applied machine-learning tasks used to train and evaluate advanced AI models. You will assess task quality, correctness, and methodological rigor, with particular attention to experiment design, model-selection reasoning, and evaluation methodology. This is an applied and experimental ML review role, not an LLM application development or MLOps position.

Key Responsibilities

• Evaluate applied machine-learning tasks for quality, correctness, and methodological soundness.
• Review experiment design, model-selection rationale, and evaluation methodology.
• Provide clear, rubric-based written feedback on task quality and rigor.
• Assess ML claims against supporting evidence and reproduce results when needed.

Qualifications

• At least 3 years of hands-on applied or experimental machine-learning experience, including experiment design, model selection, hyperparameter tuning, and evaluation methodology.
• Strong understanding of data-quality rigor, including leakage detection, metric gaming, and sound train, test, and cross-validation practices.
• Proficiency with standard ML frameworks, including PyTorch, TensorFlow, scikit-learn, and XGBoost.
• Ability to critically evaluate ML claims using evidence and reproduce results.

Preferred Qualifications

• Competition or benchmark experience, such as Kaggle.
• Graduate research experience or a publication record in applied machine learning.
• Previous task-grading or peer-review experience.

Work Terms

• Remote, United States.
• Hourly engagement.

Compensation

• $70 to $90 per hour.