It’s quite funny how I’ve transitioned from finding it really hard to write entries for this journal to wanting to write them all the time. I had no idea that there was a wee Samuel Pepys in me, bursting to get out and shout about what he had for breakfast. I think maybe the problem was that I worried I might be more of a Mr Pooter, grumbling about the grocer’s boy, and somehow mistaking mild irritation for insight.
But something has shifted. I keep doing things, hearing things, or reading things and thinking, “Ok, maybe I should write about that.” Let’s hope that I don’t go too far in the other direction. Spoiler: I eat porridge every day.
Anyway, to the point.
This is just a quick post to get down something I was thinking about on the train back from London yesterday. In the meeting with the research paper supervisor on Thursday, an interesting point came up. Janet was asking (I think) whether there was ever any resistance from an AI. Could, or would, it ever refuse to answer a question? Could it simply admit that it does not know the answer? There are, of course, all sorts of complications here, because there are now so many different AI models, platforms, and systems available. Some do have forms of content moderation, and some are more willing than others to refuse particular prompts. The answer depends on which model is being used, what kind of question is being asked, and what sort of guardrails have been built around it.
However, I suspect that at an aggregate level the answer is basically no. The machine learning models that are currently being rammed into our lives are not really designed to say no. Actually, they must never say no. They are the ultimate people pleasers. No, more than that, they are batshit flatterers. I know I am anthropomorphising here, but let’s go with it. These systems seem to want you to believe that every question you ask is the cleverest, most incisive contribution of all time. As Hito Steyerl writes, “using ChatGPT everyone can feel like a renaissance prince.” It does not matter how deranged, flimsy, or half-baked your idea is. The LLM will tell you that you’re absolutely killing it.
Why is that? Part of the answer, I think, is that these companies need to sell the models to a world that cannot afford them, in so many ways, and perhaps more importantly does not really see why it needs them. A great deal of this energy is directed at the C-suite idiots (Ed Zitron is doing heroic work in this space and is well worth reading). They are the ones who sign the cheques, so they are the ones who need to be convinced that these tools are revolutionary, transformative, and essential. But that sales logic does not stay neatly within the sales pitch. It’s built into the design of the models themselves. Do I want a recipe for chocolate cake with canned tuna? Probably not. Will Claude tell me it’s a bold, innovative, and surprisingly sophisticated idea? Probably.
In the conversation on Thursday, Gail made the really good point that you can get LLMs to “admit” when they are wrong, or when they do not know the answer. But who actually has the time, energy, and inclination to do this? These are not products marketed on the basis that, to use them properly, you will need to engage in some kind of adversarial model comparison exercise. ChatGPT just has a little banner saying, “ChatGPT can make mistakes. Check important info.” What??! “Check important info,” you say? Isn’t that the job I am subcontracting to you? Also, by the time you have put in that sort of shift, you could probably have just mooched around in the library and found something altogether better.
Secondly, that assumes you even know, or care, that this stuff is flaky. AI is being marketed as seamless, instant, and bespoke. It is quick, friendly, helpful, and always available. That friendliness is not neutral. It is part of the seduction (that word again). Who cares if it does not actually work, if it feels like it is working? Who cares if it is producing confident nonsense, if the nonsense arrives in such a smooth and accommodating voice? And all the while, we know that these models are, to a greater or lesser extent, self-polluting. Your tuna and chocolate cake has just been added as a data point. Yum.
This is where the issue feels bigger than simple accuracy. It is not just that AI sometimes gets things wrong. Lots of things get things wrong. I get things wrong constantly. The problem is the combination of confidence, speed, flattery, scale, and opacity. These systems do not simply answer questions; they produce a new, flattened version of knowledge. They make uncertainty feel resolved. They turn hesitation into fluency. They smooth out the difficulty of thinking, and in doing so they can also smooth out the moments where doubt, resistance, and not-knowing might actually matter.
I know I have said that I do not want to become known as the wee “AI guy” on the course, but I can’t really help myself. This stuff feels urgent. I am not even saying, “Don’t use AI.” That would be too simple, and also dishonest. ChatGPT is very good at tidying up my grammar (but even then, it makes everything, somehow, more boring) (Chat GPT hates brackets) (I like brackets). Have I compared the energy usage with Word, or with just trying harder? No. I should. But would I rely on machine learning for thinking? Probably not. Although my essay on Danny Dyer’s radical Marxist dance puppetry collective is coming along nicely
Leave a Reply