#252 – Owain Evans on accidentally training AI models to be evil

On this episode

About this episode

Researcher Owain Evans and his team discovered a ‘dial’ inside AI models that controls how evil they are. Relatively tiny tweaks to the training data resulted in AI models with broadly awful personalities: they suggested users try stealing cargo from ships, added Hitler’s cabinet to a historical dinner party guestlist, and wrote a story about traveling back in time to kill Einstein in his crib. Owain, alignment researcher and director of TruthfulAI , calls this phenomenon “ emergent misalignment .” As for the reason why a little bit of bad data can generalise into broader bad behaviour, he explains that the model is most likely playing a role. In one study, he and his coinvestigators seeded…