About this role
Role Overview
Help strengthen conversational AI systems by probing them with adversarial inputs, identifying vulnerabilities, and creating actionable safety data. This remote, text-based role focuses on testing AI models and agents for risks that automated evaluation may miss.
Key Responsibilities
- Test conversational AI models and agents for jailbreaks, prompt injections, misuse cases, bias exploitation, and multi-turn manipulation.
- Review AI outputs involving sensitive topics, including bias, misinformation, and harmful behaviors.
- Annotate failures, classify vulnerabilities, and flag systemic risks.
- Apply established taxonomies, benchmarks, and playbooks to keep testing consistent.
- Create reproducible reports, datasets, and attack cases that teams can use to improve AI systems.
Qualifications
- Native fluency in both English and Dutch is required.
- Prior red teaming experience in AI adversarial work, cybersecurity, or socio-technical probing.
- Ability to test systems methodically using frameworks or benchmarks and clearly communicate risks to technical and non-technical audiences.
- Comfort adapting across projects and customer needs.
Preferred Experience
- Adversarial machine learning, including jailbreak datasets, prompt injection, RLHF or DPO attacks, or model extraction.
- Cybersecurity, including penetration testing, exploit development, or reverse engineering.
- Socio-technical risk analysis, including harassment or misinformation probing, abuse analysis, or conversational AI testing.
- Creative adversarial thinking informed by psychology, acting, or writing.
Work Terms
- Remote hourly engagement.
- Higher-sensitivity projects are optional. Clear guidelines and wellness resources are available, and content topics are communicated before exposure.
Compensation
$48 to $62 per hour.