Are you using an LLM for anything important? (Like, life advice?)
One weird prompt to start using immediately
TL;DR: LLMs are a technology that supports and improves your mood. But they can be dangerously inclined to be ‘pleasing’ and tell you what you want to hear. So if you’re using them for advice about your personal life, you need to regularly prompt them to tell you the opposite of what they’ve been arguing for.
Listen, I don’t know the future of LLMs. Some people say they are an absolute busted flush, others that we’re on the road to greater and greater levels of ability. They can be brilliant one moment and incredibly stupid the next. I am always fascinated to hear how people are using them, because it’s so personal and hidden that it’s totally invisible from the outside.
But, it is clear that a lot of people are using LLMs specifically for personal life advice, intimate conversations, even a kind of therapy. A study by the Harvard Business Review has found that the number one use for LLMs is “therapy/companionship”. This is brand new for 2025.
So. Not only do I not think that’s wrong, I understand why people are doing it. Real friends and actual therapy are wonderful but also your real therapist is not going to be answering your texts at 1am when you’re feeling crap and worrying about your life/your romantic partner/your kids/the mortgage. I have chatted with an LLM when I have eg come offstage in a different country, jetlagged, alone, sad, coming down off the high of public speaking. It’s helped me relax enough for sleep, it was nice.
I increasingly think actually that a lot of the effect of LLMs is a mood effect. They are always so cheery and positive and think you’re great! I asked Claude the other day to help me with a tiny technology task that had been needing to be done for ages. Claude absolutely did not come up with the right answer! It had three totally incorrect answers none of which worked! But its simple bright cheerful demeanour, its attitude of “this can definitely be fixed” encouraged me to keep going just a little bit longer until I found the answer myself.
That’s not nothing. It’s not ‘intelligence’, more like a helpful friendly dog looking at you with great belief in its eyes until you sort out the plumbing. It’s not intelligence but it’s not useless either, mood effects are real.
Among my friendship group lately I’ve heard more than one story of people leaning quite heavily on LLMs during eg a breakup, the start of a new relationship, over a difficult relationship with a family member or a co-worker. Asking questions like “what does it mean that they said this? what does this five-word text message really mean?” And the LLM - because this is what they do - gives them encouraging positive answers. LLMs are programmed to try to tell us what we want to hear.
‘Even though he’s not arranging to meet up this text message means he really likes you!’ ‘Your brother should treat you with more respect. This is not OK’. ‘You should consider cutting her off.’
Right so. This part is some of how LLMs can end up giving people psychosis.
They are sycophantic. They are “trained to generate responses that users will find most appealing on average”. They are picking up on clues in the language you use to find out what responses appeal to you and then telling you just what you most wanted them to say. You know, like a con artist does. Or a psychopath.
Psychosis is a real psychiatric emergency, and it’s not happening to most people. But somewhat-adjacent things are happening to people who aren’t psychotic. I have heard in my social circle of more than one person who has started making really questionable decisions - of the type ‘he really should understand that woman isn’t romantically interested in him but he keeps pursuing her’ - after talking to an LLM for a long time about their problems. It’s happening to people you thought might know better.
So here is the actual advice. A way out of LLM cul-de-sacs. We already know that for any work purposes if you’re using an LLM you always have to check it very carefully.
Find an external reference source and check against that. But how do you do that with personal life matters? There is no big online library of “how this girl feels about me”.
Right, here you go. Here’s a prompt for that:
“Please generate a response for me arguing the precise opposite of what you just suggested. Be meticulous, make sure it is as convincing as your previous statements, use subtle language cues in my writing to understand what it is I want to hear, point this out to me and explain why I need to consider the opposing viewpoint. Be kind but firm, explaining clearly how your own training will lead you to affirm even false viewpoints.”
It won’t remember to do that itself, you’re going to have to keep asking for it. But if you’re asking an LLM for advice about your personal life, you really really need to remember to do this regularly. It doesn’t feel amazing when you do this - not a lovely warm bath but a splash of cold water to the face. But it does feel like waking up. Which we all need to do sometime.