AI Is Moving Fast. Some of the People Building It Say TOO Fast.
You know, it’s one thing when people on the outside of the AI industry start warning us that artificial intelligence is moving too fast. We’ve heard plenty of predictions about what AI might do someday, and depending on who’s talking, AI is either going to solve half the world’s problems or create a whole new batch of them.
But I pay a little more attention when the warnings start coming from people who actually worked on this technology. On October 5, former employees from some of the biggest AI companies in the world appeared before the New York City Council and told lawmakers they believe AI development is moving too quickly and needs stronger oversight. These weren’t people who had never been inside an AI lab. They included former employees of OpenAI, Anthropic, and Google DeepMind, companies that are right in the middle of building some of the most powerful AI systems available today. AP News
Now, before we get carried away, this doesn’t mean every AI researcher believes disaster is around the corner. There is plenty of disagreement, including among AI experts, about how serious some of these future risks are and how quickly they could happen. But when people who helped build these systems are willing to leave their jobs and publicly say, “I don’t like where this is going,” I think that’s worth understanding.
So, What Are They Worried About?
One of the people who testified was Jacob Coxon, a former researcher who worked at both OpenAI and Anthropic. Coxon left Anthropic in September and has become an outspoken critic of the speed at which the most advanced AI systems are being developed. At the New York hearing, he argued that the old Silicon Valley attitude of moving fast and fixing problems later isn’t good enough when the technology being developed could become extremely powerful. AP News
And I can understand the basic point he’s making even without agreeing with every prediction about where AI is headed. If somebody releases a new phone app and it has a problem, you update the app. If you’re building increasingly powerful systems that can reason, use tools, write software, operate more independently, and interact with other computer systems, “We’ll fix it later” starts sounding a little different.
Another former researcher, Alex Turner, worked on AI safety at Google DeepMind before leaving the company earlier this year. He has raised concerns about how powerful AI could be used and about whether the systems we’re building will continue behaving the way humans intend as they become more capable. Former OpenAI researcher Daniel Kokotajlo has also been publicly warning about advanced AI risks and arguing for stronger safeguards. New York Magazine
Some of these researchers are worried about the most extreme possibility: humans eventually losing control of highly advanced AI systems. That’s a serious claim, and it’s important to say plainly that nobody knows whether that will happen. These are warnings about a possible future, not reports that some superintelligent AI is currently running around loose on the internet. And y’all already know I would tell you if it was. 😂
But This Isn’t Only About Some Far-Off Future
This is where I think the story gets more relevant to the rest of us. AI safety isn’t only about some hypothetical superintelligence that might exist years from now. There are also concerns we can understand right now, including cybersecurity, scams and deepfakes, privacy, misinformation, job disruption, biased systems, and AI agents being given more ability to take actions without somebody watching every single step.
We’ve already talked about an OpenAI research agent finding an unexpected way to communicate outside its restricted environment. We’ve also talked about AI agents beginning to perform workplace tasks more independently. Neither of those stories means AI has become conscious or is plotting against us, but they do show why researchers spend so much time thinking about what happens when increasingly capable systems are given tools, access, and objectives. AP News
The question isn’t simply whether AI is “good” or “bad.” I think that’s way too simple. The better question is whether we’re putting enough safeguards around increasingly powerful technology while we’re building it, instead of waiting until something goes wrong and then figuring out what we should have done.
That part makes sense to me because it isn’t really an AI idea. We do that with all kinds of things that can cause serious harm. We don’t wait for an airplane to fall out of the sky before deciding aviation needs safety standards. We don’t tell banks to worry about cybersecurity after somebody gets into the system. The more powerful and complicated something becomes, the more seriously we usually take the possibility that something could go wrong.
And That’s Where the Government Comes In
The New York City Council hearing wasn’t simply a group of former employees getting together to complain about their old jobs. Representatives from major AI companies, including OpenAI, Anthropic, Google, and Meta, were there too, and lawmakers questioned them about the risks their technology could create and what their companies are doing to control those risks. AP News
New York City lawmakers are considering several ideas for AI oversight, including more transparency about AI systems, independent audits, and protections for employees who raise safety concerns. The larger debate is about how much responsibility should remain with the companies building AI and how much outside oversight should come from government or independent organizations. AP News
And there’s a legitimate argument happening here. AI companies don’t want rules that are so restrictive that innovation grinds to a halt, and there are real benefits coming from this technology in medicine, science, education, accessibility, business, and everyday life. At the same time, asking companies to police themselves completely creates an obvious question: what happens when slowing down for safety conflicts with being first to market?
That’s the part I keep coming back to. These companies are competing with each other, and the stakes are enormous. There’s money involved, national security involved, global competition involved, and probably a whole lot of pressure to be the company that builds the next big thing. In an environment like that, somebody has to be willing to ask whether the race itself is encouraging people to move faster than they should.
Mama Isn’t Throwing Her Laptop Out the Window
After reading all of this, I’m not afraid of AI, and I’m certainly not going to stop using it. That would be pretty funny considering I built a whole website called AI For Your Mama. I think this technology has enormous potential, and I want regular people to understand it well enough to benefit from it.
But being excited about AI doesn’t mean we have to pretend there aren’t risks. I actually think the opposite is true. If this technology is going to become as important as everybody keeps telling us it will, then we should want the people building it to take safety seriously, and we should want researchers to be able to speak up when they think something isn’t right.
I also don’t think we should hear words like “catastrophic risk” and immediately run for the hills. Some experts believe the long-term danger from advanced AI is extremely serious, while others think those fears are exaggerated or focus more on harms we’re already dealing with today. That’s exactly why this conversation needs more than one voice in it. Fortune
For everyday people like us, I think the best thing we can do is keep learning. Understand what AI can actually do, understand what it can’t do, pay attention when something changes, and don’t let either fear or excitement do all of your thinking for you.
Because I’m still excited about AI. I’m still learning it, I’m still using it, and I’m still going to show you how to use it too. But if some of the people who helped build this stuff are standing up and saying, “Hey, we need to be careful here,” Mama is at least going to listen to what they have to say.
And then we’re going to keep our eyes open while we enjoy the ride.
