Skip to content

ML Researcher, Post-Training and Safety

Compassion Aligned Machine LearningRemote

View the jobThis takes you to the employer’s own site, where you apply.

Global catastrophic risksAI safetyResearchSoftware engineering

Why this is on the board

If you teach a language model certain values, do they hold up when someone tries to mislead it or trains the model further? What you discover about this using open models is published openly, so that other researchers and AI companies can check it and adopt it, even if it turns out that a common approach falls short.

This board is in beta and this is our judgement. If you think we have it wrong, tell us. Give feedback · How we reason about it

About the role

Run LLM training experiments on value instillation through mid-training data and post-training RL, then evaluate whether those values persist. - Design and execute RL post-training (RLHF, DPO, GRPO) and mid-training synthetic data experiments on 12B-scale open-weight models. - Build evals to measure value persistence under adversarial prompting, distribution shift, and training-induced degradation like reward hacking. - Write up results for arXiv and LessWrong; flag broken findings and disagr…

More about Compassion Aligned Machine LearningWhy this problemGlossary

Not the right role? Get new ones every month

On the first Tuesday of every month: new jobs and programmes from this board, plus events in the next six weeks and the best of the EA Forum.

Which language?

We’ll email you a link to confirm first. Unsubscribe any time.

Want to understand this first?

Start with our free course: about 100 minutes in six short units, at your own pace, with no sign-up. To go further, apply for the EA Fellowship: five weeks of reading and discussion in a small group. Also free, but selective.

Disagree with something here?

This board is curated by people who can be wrong. If you think a listing does not belong, if we are missing one, or if something else is off: Give feedback, or email jobs@effectiefaltruisme.nl.

Choose which cookies you allow. Necessary cookies are always on because the site can't work without them.

Always on

Keep the site secure and remember your cookie choice. These don't track you.

Let Google Analytics and Microsoft Clarity show us how people use the site, such as which pages they visit and how they move around a page, so we can improve it.

Let Google and Meta measure whether our ads, such as for the EA Fellowship, reach the right people.