The Broom Problem

There's a story making the rounds that I keep coming back to. As it's been reported, OpenAI spent about a million dollars' worth of compute pointing its models at Linux, the operating system that runs most of the world's servers, phones, and a lot of things in between, and came back with nearly a thousand exploitable holes. Linux isn't some weekend project. Some of the most careful engineers alive have been reading and arguing over that code for more than thirty years. The machine found what they missed, and it found a lot of it.

My first reaction was admiration, and I think that's the right first reaction. Finding flaws is hard, tedious work. It takes patience most of us don't have and attention we can't hold for long. If a machine can sit with a codebase and read it more carefully than any team of people, then the software we depend on can get safer. That's a real good, and I don't want to rush past it.

My second reaction came a little later and stayed longer. Whatever can find a thousand holes can find a thousand more, and it doesn't care who's asking. The same capability that helps a maintainer patch a flaw helps someone else use one. You can't build a tool that only sees weaknesses when the right people are looking. Once the ability exists, it's out in the world, it gets cheaper, and it spreads. Every advance in finding problems is also an advance in exploiting them. The sword cuts both ways, and it gets sharper every year.

I don't think that's a reason to stop. I do think it's a reason to be honest about what we're doing.

The speed of it

The same double edge shows up when you move from software to work.

People like to point to the industrial revolution when they want to calm everyone down about jobs. Machines replaced farmhands and weavers, the argument goes, and we came out fine. We did, eventually. But that change played out over the better part of a century. A farmer's son went to the mill, and his son went to the factory, and his son went to the office. Each generation had time to learn new work, and the economy had time to invent jobs nobody had imagined yet. The absorption was slow, and it hurt, but it was slow enough for people to adjust.

This time the change isn't moving at the speed of generations. It's moving at the speed of software updates. A job that was safe in January can be in question by June. A person who spent fifteen years getting good at something can watch it become a feature. We've never tested whether people and institutions can absorb that kind of change inside a single working life, much less inside a few years.

For now, a lot of work still needs a human being. Somebody has to walk the rail line and check the ties. Somebody has to crawl under a house to fix a broken pipe, or open a panel and figure out why the breaker keeps tripping. Somebody picks up the trash at six in the morning. Somebody turns a patient in a hospital bed, sits with an old man in a nursing home who doesn't remember his daughter's name, and helps a family through the worst week of their lives at a funeral home. That work is physical and specific and full of judgment calls, and machines aren't good at it yet.

I notice I keep saying "yet." Robotics is behind software, but it isn't standing still. It's fair to ask what happens when the robots can do the plumbing and the lifting and the night shift too. I don't mean that as a scare line. It's a question we'll have to answer, and I'd rather start thinking about it before it arrives than after.

What work is for

The easy answer is that people will be freed up. Machines do the work, the gains get shared somehow, and we all have more time. Maybe. But I've watched what happens to people when structure disappears, after a layoff, a retirement that came too early, or a long illness. Free time isn't the same thing as freedom. Without something to get up for, a lot of us drift. Days lose their shape. Purpose thins out, and something worse tends to fill the space.

There's an old idea in Genesis that I think gets overlooked. Before anything went wrong in the garden, the man was put there "to work it and keep it." Labor wasn't the punishment. The punishment came later, and it was the sweat and the thorns. The work itself was part of the original design, part of what a person was made for. You don't have to share the faith to see the insight. People need something to tend. We need to be useful to someone, to make something, to have a reason the day matters.

So the question isn't only whether there will be enough jobs. It's whether there will be enough purpose, and whether we'll have the sense to protect it when the machines can do almost everything for us.

Knowing where you're weak

I don't have a policy answer for any of this. What I keep landing on is something smaller and older, which is self-awareness.

The best people I've worked with aren't the ones without weaknesses. They know exactly where theirs are, and they route around them. They know they get impatient late in the day, so they don't make big decisions at five o'clock. They know they're bad with details, so they ask the person who isn't. They don't pretend. Pretending is how you end up blindsided by the thing everybody else could see coming.

That holds for a person, and I think it holds for a society working through this technology. The dangerous position isn't "we have weaknesses." It's "we're sure we don't." It's the confidence that we'll figure it out, that it'll sort itself out the way it did last time, and that the people building these systems surely have it handled. That isn't optimism. It's self-delusion. If AI is good at anything, it's finding the holes we didn't want to look at. We could stand to point a little of that honesty at ourselves.

The broom

When I try to put words to what people are actually afraid of, I don't think of the movie robots. I think of Mickey Mouse.

In Fantasia, Mickey is the sorcerer's apprentice. He's tired of hauling water, so when the old sorcerer leaves he puts on the hat and brings a broom to life to carry the buckets for him. It works beautifully, at first. Mickey dozes off and dreams of commanding the stars. When he wakes up, the room is flooding, and the broom is still carrying water, because nobody told it when to stop and Mickey doesn't know how.

So he does what a panicked person does. He grabs an axe and chops the broom to pieces. For a moment it seems to work. Then every splinter stands up, grows arms, picks up a bucket, and marches back to the well. One problem becomes a hundred. His fix made it worse, and the water keeps rising.

I think that's the real fear. Most people don't expect the machine to turn evil. They're afraid we've started something we don't fully understand, that it will keep doing what we asked long after we wanted it to stop, and that our attempts to control it will multiply it instead. The thousand holes in Linux, the jobs moving faster than people can follow, the drift that comes when the work is gone, all of it feels like water rising around an apprentice who knows half the spell.

Notice that Mickey's problem was never the broom. It was that he reached for power he didn't understand, out of a very ordinary wish not to carry the buckets himself, and then he wouldn't admit what he didn't know until the room was underwater. In the film, the sorcerer comes back, raises his hands, and the water goes away. I'm not sure we get a sorcerer.

What we do get is the chance to be a better apprentice. We can learn the whole spell before we cast it. We can be honest about where we're weak. And we can keep carrying some of the buckets ourselves, because the work was never only about the water.