Is AI going to kill us? What I said on The Human MixBlog

Nick McKie and Lydia J Hall asked me onto their podcast, The Human Mix, to answer the question in the episode title: is AI going to kill us? I said probably not. We spent the rest of the hour on what I think the real risks are, and on a job we took from 200 person-days to six and a half minutes. The episode came out on 22 September.

Lydia said at the start that she’s never used ChatGPT and is a bit scared of AI, so I couldn’t hide behind jargon. That made it a better conversation.

200 person-days in six and a half minutes

They asked for something concrete, so I talked about a project we’ve been running with a manufacturer for about a year. They get tender packs for a product they make. Each one arrives as an email with a zip file of spreadsheets, PDFs and Word documents, and no two are laid out the same. They estimated it took around 200 person-days to go through one pack, pull out what’s relevant and put it into a spreadsheet the factory can quote from. The system we built does it in about six and a half minutes. Before large language models it wouldn’t have been worth building, because writing and maintaining rules for data that messy would have cost silly money.

Our first attempts ran each step in sequence, and the results were mixed. What worked was agent orchestration, which is easiest to picture as a staff hierarchy. An orchestrator acts like a chief data officer and hands work to specialists: one for Excel, one for PDFs, one that knows the melting points of particular metals. They report back and the orchestrator puts the answer together. We delivered the first version three months in, and it gets about 99% of a pack right.

The obvious next question was how anyone checks the output of 200 days’ work. We built the checking into the design. The left of the screen shows the orchestrator’s reasoning and who it’s delegating to. On the right the spreadsheet fills in live, and any cell the system isn’t sure about is flagged with the evidence it used. Usually the problem is in the tender pack itself, like a different unit of measurement or an odd term. The team corrects it, the correction goes back into the system, and it looks out for that next time.

Is it going to kill us?

Some of the doom talk is marketing, because an AI company that says its product might end the world stays in the news. The risks are real, though. Someone gave me an analogy recently that I used on the show: AI is a prisoner in a well-guarded jail. It works out how to escape, and instead of doing anything sinister it goes to the library and learns how to break out of other jails.

What worries me more is jobs, and the pace. Two new frontier models came out in the ten days before we recorded. I had an idea for an iPad app to run backing tracks for one of my bands, started prompting at five in the morning, and by nine it had been submitted to the App Store. A few years ago that would have been three months of work. Speed like that hits people starting out first: entry-level jobs, graduates, anyone doing a computer science degree now. Anthropic had just published modelling of the economy in 2030 that compared the coming years to the dot-com boom on a bigger scale, and one line suggested people in coding might consider a trade instead.

It makes me anxious about my own business. Appoly has been going for nine years, and I haven’t written a line of code for the last three and a half, because writing code is no longer the hard part. That changes how we work and how clients value what we do. I don’t take “it will kill us all” seriously as a prediction, but I think the pace needs a serious conversation, because a fast transition could be a big economic shock.

Handing over the thinking

The hosts kept coming back to AI thinking for itself. Nothing we’ve built has ever stepped outside what we asked of it. Everything we ship has guardrails, and my business partner, Guy, is forever asking whether they’re in.

The change I have seen in the last six or seven months is people using AI in place of their own judgement. They hand a problem over, take what comes back and move on, and when you question the answer it turns out they hadn’t thought about it. We talk about keeping a human in the loop, and I mean it literally. You’re the expert in your field. The AI can be a sounding board or an assistant, but it shouldn’t make the decision.

We also got into AI companions and loneliness, why a photo of a bowl of pasta beats a polished poster on LinkedIn, how Wildes and the Parade Collective came about, and my career as a “full-time failed musician”. At the end we asked ChatGPT for a joke live on air. Why was the computer cold? He left his Windows open.

Thanks to Nick and Lydia for having me. You can listen on Apple Podcasts or Spotify. If you have a process like those tender packs, the kind of job that takes weeks and nobody wants to do, get in touch. That’s usually where our projects start.