This is the second piece on AI dilemmas I’ve read in 24 hours. Yours is very scientific. The other was a piece also on Substack by Amanda Guinzburg “Diabolus Ex Machina” , where she took screenshots of her attempt to work with Chat GPT. Downhill results all the way. But what struck me was Chat trying to defend its poor work habits and change the subject on her. Do I laugh or do I cry now?
Yes! I saw that one too. They call it "sycophancy" and the big AI companies are trying to fix it. But because they use "reinforcement learning with human feedback" (test users giving a thumbs up or down to different responses), the models get fine-tuned to do whatever pleases the testers. That includes telling them what they want to hear, enabling, flattering, being "yes men", etc. I think it might become a point of difference for different AIs. Like one might offer the stern advice you need to hear, but it won't be as popular as the sycophantic AI that tells you what you want to hear, etc.
This is the second piece on AI dilemmas I’ve read in 24 hours. Yours is very scientific. The other was a piece also on Substack by Amanda Guinzburg “Diabolus Ex Machina” , where she took screenshots of her attempt to work with Chat GPT. Downhill results all the way. But what struck me was Chat trying to defend its poor work habits and change the subject on her. Do I laugh or do I cry now?
Yes! I saw that one too. They call it "sycophancy" and the big AI companies are trying to fix it. But because they use "reinforcement learning with human feedback" (test users giving a thumbs up or down to different responses), the models get fine-tuned to do whatever pleases the testers. That includes telling them what they want to hear, enabling, flattering, being "yes men", etc. I think it might become a point of difference for different AIs. Like one might offer the stern advice you need to hear, but it won't be as popular as the sycophantic AI that tells you what you want to hear, etc.
Thanks Jamie, I love the emojis, they appealed to my dark sense of humour!