The Coding Rollercoaster: AI Development Reality

About this post

We examine why AI-assisted software development oscillates between extraordinary productivity and maddening dead-ends, why that volatility is intrinsic rather than incidental, and how to build a working practice that absorbs it instead of being derailed by it.

Key takeaways

The same model, on the same task, does not perform consistently - and the swing is largest exactly where you can least afford it. Treat AI as a fast, brilliant, and unreliable collaborator: keep the speed, distrust the output, and let human judgment remain the integrating function. Match the model and the tool to the task rather than staying loyal to one.

The original ideas, structure, and much of the language are human-created, but AI was used to develop, enrich, or rework portions of the content — for example, researching sources, rewriting sections for clarity, or expanding on arguments.

Recently, I needed to convert an Excel-based tool into something more robust, and I decided to build a React.js application with the help of Claude Opus. Within less than four hours, I had a flawlessly working web application, just based on Claude’s analysis of the Excel structure and content. I was obviously super-happy, praising a tool that helped me do something that would have cost me at least $10,000 if I had hired an offshore developer to build it, not counting the numerous hours of back and forth during debugging I would have spent.

The next two days, however, I spent coaching Claude Opus debugging the positioning of headers and footers in the PDF output of that new tool. I was frustrated and ready to throw in the towel. Had I done it myself or hired a freelancer, it would have taken half an hour at best.

Since I started using AI to develop software, these ups and downs have become an essential part of my life.

What I want to unpack here is why the experience swings so violently between miracle and misery – because the reasons are structural, not bad luck – and what actually changes once you stop expecting the tool to behave like a consistent colleague and start treating it like what it is.

The miracle is real – and so is the misery

Let’s begin with the good half of my story, because dismissing it would be a mistake. The four-hour React build was not a fluke or a toy. AI genuinely collapses the cost of the work most developers find tedious: reading an unfamiliar data structure, scaffolding an application, wiring up boilerplate, translating intent into a first working draft. In that mode it is not a little faster than a human – it is faster by an order of magnitude, and often cheaper than you can quite believe.

AI gets you 80% to an MVP; the last 20% requires patience, learning deeply or hiring engineers.

Addy Osmani, Director Google Cloud AI, 2026

What the miracle of quick success quietly trains you to do: it teaches you to trust. And the tedious two-day stretch to fix something seemingly obvious (the size of a logo and its positioning) is not the worst flaw. I remember days where I was inattentive after a good moment and the AI model destroyed more than it fixed, by taking shortcuts, covering up mistakes, and creating bloated code with patches over patches. The effect: I reverted every change made that day and grudgingly accepted that a lot of work was wasted.

In all these cases the results were presented with conviction, and only my scrutiny stopped this from bleeding into the codebase. “Now I have the full picture” is only one of the famous misleading sentences an AI model uses, suggesting confidence when there is none.

Here’s the first thing to internalize: the high and the low are not separate experiences. The high creates the conditions for the low. The smoother the early wins, the less you scrutinize – and the harder the eventual wall hits.

The key issue: nothing about AI output is stable

AI writing code:
strengths and weaknesses

The story in the introduction points straight to where AI fails most when coding – and it comes down to one thing: how much solid example code it had to learn from.

Think of it as the market for developers. If you want someone to build a straightforward React app, there’s no shortage of candidates. Post on any freelance platform and you’ll have a stack of qualified people, with references, within hours. But if you need someone to fix a bug in one particular PDF-generation library, you can search for days – and you’ll mostly turn up people who claim they can solve it, and very few who can prove it.

LLMs embody exactly this reality. Point one at a common problem in a common tool, where it has absorbed a vast swath of examples, and it will likely deliver solid code, fast. Push it into a niche, a boundary case, or a thorny debugging session, and the ground falls away. It doesn’t stop and tell you it’s out of its depth – it keeps generating with the same confidence as before, improvising, hallucinating a plausible fix, or quietly patching over the problem to give you what you asked for: something that looks like it works.

That’s the whole pattern in one line: AI is weakest exactly where there is the least knowledge to draw from – and it never sounds any less sure of itself when it gets there.

To understand the swings, you have to drop the intuition that you are working with a colleague who has a consistent skill level. You are not.

A Large Language Model does not have a fixed competence the way a person does. Each response is generated fresh, shaped by a cascade of probabilistic choices over the context it currently holds. The same model, given the same problem, can land on an elegant solution one session and flail through ten broken attempts the next – not because it “knows” less, but because this particular draw of the deck went differently. The fluency stays constant whether it is on solid ground or improvising; the reliability underneath does not.

AI is most reliable where its training data is densest – common frameworks, popular libraries, well-trodden patterns. As soon as there is less available knowledge, the output silently shifts from recall to invention. The general React scaffold sits in the dense center of what the model has seen a million times, the PDF formatting bug in the sparse periphery. That is not a coincidence; it is the pattern. The tasks that feel small to you are often the ones sitting exactly where the model is weakest.

Today is not like any other day

The model itself is a moving target. You might assume that picking a specific model locks in a known quantity. It does not. Providers run fleets of servers with different hardware and quantization levels, and they silently change routing, default reasoning effort, system prompts, and caching. Between March and April 2026, a wave of developers reported that Claude’s Sonnet and Opus models had gotten noticeably worse – one engineering director audited 6,852 of her own sessions and over 234,000 tool calls and showed the model’s reasoning depth collapsing – and the provider’s own postmortem confirmed real causes: a change that lowered the default reasoning effort (later reverted), a session-handling bug that made the model forgetful and repetitive, and a verbosity-trimming prompt tweak that degraded answers. The weights hadn’t rotted – the models themselves and the API were confirmed unaffected – but the experience genuinely degraded, for reasons no user could see. So part of a “bad day” may have nothing to do with you and how you prompted the model. It is the machinery around it, shifting under your feet like quicksand.

Under this lens, volatility stops looking like bad luck. It is the expected behavior of a probabilistic system whose reliability is unevenly distributed and whose infrastructure shifts daily.

The numbers behind AI coding

The unevenness is measurable. AI is measurably good at the routine layer of coding – and measurably worse than humans at the deep layer. Let’s start with the good half, because it’s real. An Apiiro analysis of AI-assisted development by large enterprises found that the tools sharply cut the standard failure modes: simple syntax mistakes dropped by around three-quarters and routine logic bugs by roughly 60%. AI really does reduce the pain.

One layer down, the picture inverts. A few numbers on that:

In Veracode’s 2025 GenAI Code Security Report, which put more than a hundred models through standardized security tasks, about 45% of the generated code samples introduced a vulnerability from the OWASP Top 10 – the most common, best-documented web flaws there are, things like SQL injection and cross-site scripting. These are not exotic edge cases the model couldn’t have known about; they are the failures every security course teaches first. The weakness clustered by language, too: Java code failed roughly seven times out of ten.

The second is that this is not improving on its own. When Veracode re-ran the exercise for its March 2026 update, the pass rate had barely moved – hovering around 55% even as the same models posted higher and higher scores on coding-skill benchmarks. That gap is the single most important number here: the models kept getting visibly better at coding while staying flat on writing safe code. Capability and safety are not the same axis, and only one of them was climbing.

The third is what all this accumulates into. A 2026 analysis of AI-authored commits across thousands of public repositories tracked how much unresolved technical debt was surviving in the codebase over time, and found it climbing from a few hundred lingering issues in early 2025 into the hundreds of thousands within about a year. This is the slow cost, the one you don’t feel on any single good day: volume generated fast, with the deeper problems left quietly in place.

Underneath all three sits a human number that ties the section to the rest of this piece. By one 2026 industry survey, 58% of developers said they trust AI output without testing it. That is the mechanism by which the findings above actually reach production. The defects are only half the problem; the misplaced confidence is the other half.

Sources: Apiiro enterprise code analysis, 2026 (routine-error reduction); Veracode 2025 GenAI Code Security Report and its March 2026 update (OWASP vulnerability and pass-rate figures); a 2026 analysis of AI-authored commits across public repositories (technical-debt accumulation); 2026 developer survey (trust-without-testing figure). Figures are approximate and methodology varies between studies.

Three ways of coding with AI

When developers run into this volatility, we see three responses. Only one of them is sustainable.

The Believer

The Believer is seduced by the good days. After enough four-hour miracles, they stop reading the output, accept fixes they don’t understand, and let AI-generated code flow into the codebase on trust. They ship fast and feel productive – right up until a confident, plausible, wrong change introduces a defect nobody caught, because nobody was really looking. The Believer mistakes fluency for correctness, and the bill arrives later, in production.

The Quitter

The Quitter is broken by the bad days. After the two-day formatting bug, they conclude the tool is fundamentally unreliable and either abandon it or relegate it to trivial tasks. They are reacting to something real – the volatility is maddening – but they over-fit to the outlier. By treating the worst session as the verdict, they walk away from the order-of-magnitude gains that are equally real. They throw out the four-hour miracle to avoid the two-day slog.

The Pilot

The Pilot treats the volatility as information and builds a practice around it. They keep their hand on the controls at all times: reading the code, distrusting fixes that look suspiciously easy, watching for the tell-tale shortcuts – silenced errors, deleted tests, invented libraries, hardcoded values that fake a passing result. When an agent starts flailing, they don’t argue it into compliance over twenty messages; they start a fresh session, switch models, or – crucially – recognize the task can’t be done with AI.

How to code better with AI

A few ground rules for coding successfully using AI models:

Don’t let a good streak lower your guard. The most dangerous moment is right after a run of flawless output, because that is when you stop reading. Review AI-generated code to the same standard as human code – the same tests, the same scrutiny – regardless of how smooth the session has been.

Suspect the easy fix. When a hard problem resolves suspiciously fast, check whether it was actually solved or merely hidden. A bug that “disappears” the moment a test is weakened or an exception is swallowed has not been fixed.

Treat the model as a moving target. Pin versions where you can. Keep sessions short and start fresh when quality drops – a long, polluted context degrades output on its own. Before assuming the failure is yours, glance at the provider’s status page. Keep a fallback model ready to switch to.

Match the tool to the task. Quality across the leading models has largely converged, so the bigger lever is routing, not loyalty. Send the hard, ambiguous, multi-step work to a frontier model; run everyday traffic on a cheaper daily driver; and lean on an integrated assistant or a terminal agent depending on whether you’re doing inline edits or large autonomous changes. The cost of integration friction usually dwarfs the model-quality difference for typical work.

Keep human judgment as the final integrating function. The AI can write the code, propose the fix, and explain its reasoning with great fluency. What it cannot reliably tell you is whether the result is actually correct, actually secure, and actually right for your specific case – especially out in the periphery where your trickiest problems live. That judgment is still, as it has always been, yours.

When I code with AI, I no longer expect a smooth ride, and that single adjustment changed my relationship with these tools more than any prompt technique ever did. The four-hour miracle and the two-day formatting bug are not the good and bad versions of the tool. They are the same tool, behaving exactly as a fluent, probabilistic, inconsistent system behaves – great in the dense center, and shallow on the edges, and quietly shifting underneath you the whole time.

The developers who thrive with AI are not the ones who found the perfect model or the magic prompt. They are the ones who stopped being surprised by the swing, kept their hands on the controls, and learned to fly the thing on the good and bad days.

References and further reading

Research and Industry Reports

Anthropic. (2026, April 23). An update on recent Claude Code quality reports.

Apiiro. (2025). Research on security risks in AI-generated code.

Veracode. (2025). GenAI Code Security Report.

Commentary

Laurenzo, S. (2026, March). Public audit of 6,852 Claude Code sessions and over 234,000 tool calls.

Osmani, A. (2026, January). The 80% problem in agentic coding. addyosmani.com.

The Coding Rollercoaster