Summary
- Nubank, Bradesco, Amazon and Microsoft published big AI code-generation numbers. METR measured experienced developers 19% slower. Neither side is lying.
- The numbers measure different things: the cases measure the whole system; the studies measure a developer with the tool and the rest of the workflow untouched.
- What actually changed: generating code became cheap, and the bill moved to review. The bottleneck changed places.
- The three cases share one thing, and it isn't the model: it's what they did with verification. The yardstick for your team is in section 4.
1. The numbers that don't fit in the same sentence
On one side, the published cases. On the other, the measured studies. Worth putting both on the table before picking a side.
1.1 The side that accelerates
Nubank. Migrating the ETL core — an 8-year monolith with millions of lines — was estimated at a year and a half. With the Devin agent, it shipped in two months, with a 12x efficiency gain in engineering time, according to Cognition. Nubank described the experience on its engineering blog.
Bradesco. Internal Copilot with a reported 40% efficiency improvement and teams generating up to 10,000 lines of code a day.
iFood. More than 60% of the code produced at the company is already AI-assisted, per the GenPlat technical write-up — the case that opened the AI gateway post.
Amazon. The most concrete case of all: 30,000 applications migrated from Java 8/11 to 17 with Q Developer, 4,500 developer-years saved and $260 million a year. An upgrade that took 50 developer-days dropped to hours.
Microsoft and NVIDIA. Nadella said up to 30% of Microsoft's code is now written by AI; NVIDIA made Cursor its standard and reports 3x code volume per developer.
1.2 The side that brakes
The METR study is the year's most cited, with reason: a randomized experiment, 16 experienced developers, 246 real tasks. Result: 19% slower with AI — while believing they were 20% faster.
And here's the part almost nobody cites: METR itself updated the study in February 2026. With late-2025 agents, the effect landed near zero, with wide confidence intervals — and they acknowledge a selection bias in the original design: 30 to 50% of invited developers declined to participate if it meant working without AI. The honest reading isn't "AI slows you down": it's "the measured individual gain is far smaller than the perceived one, and far smaller than the cases claim".
The Stack Overflow survey of 49,000 developers completes the picture: adoption rose from 76% to 84% while trust fell from 40% to 29%. More developers distrust the accuracy (46%) than trust it (33%), and the number one frustration, cited by 64%, is the "almost right" answer — the most expensive kind to review.
And the code that stays: an April 2026 study of 302.6k AI commits across 6,299 repositories found more than 15% of commits introducing at least one issue — across every assistant analyzed — and 22.7% of the issues still alive in the repositories' latest versions.
2. The bottleneck changed places
The two sides measure different truths because the software flow has two halves: producing code and trusting it. AI industrialized the first and didn't touch the second.
Before, writing was the expensive part. Review kept up because production was the limit: the whole team couldn't generate PRs faster than it could read them.
Now generation is nearly free, and what lands in the review queue is more volume, in bigger PRs, written by a tool that fails in a specific way: plausibly. Review catches the gross error; Stack Overflow's "almost right" demands reading with the attention of someone who wrote it — without having written it.
This is where the concept Addy Osmani, an engineering leader at Google, named in March comes in: comprehension debt — merged code nobody on the team can explain. The 22.7% of never-fixed issues in the arXiv study is that debt showing up in the data: nobody fixes what nobody understands.
The developer in METR's experiment sat exactly in that position: the tool generates fast, and he pays the verification bill alone. The balance comes out near zero. It's not the tool that fails. It's the workflow that didn't change with it.
3. What the cases did differently
Look at the three cases again, now paying attention to what is not code generation.
At Nubank, Devin was trained on examples of migrations the engineers had already done by hand, and every change went through an engineer's review and approval. The task was narrow and verifiable: a migration with a known shape, not an open-ended feature.
At Bradesco, Copilot adoption came with extra layers of review, compliance and security — which is what makes "10,000 lines a day" an operable number at a bank instead of a risk headline.
At Amazon, the Java transformation ran in a pipeline with tests: the acceptance criterion was built into the process. 30,000 applications don't get through manual review on goodwill.
The pattern is one: all three accelerated generation and industrialized verification. Narrow task with a clear criterion, review with an owner and a rule, automated tests wherever possible. The gain didn't come from generating more code; it came from being able to trust the generated code faster.
The difference between the case and the study isn't the model, nor the team's talent. It's that in the case, verification is a process with an owner — and in the experiment, it's one developer paying that cost alone.
4. The yardstick: three questions to know which side you're on
As in the gateway post, these are questions you answer in one meeting. If you need to ask around, that's already a data point.
4.1 Has your average PR size grown since AI came in?
A bigger PR is the first symptom of the bottleneck moving: generating got easy, and the cost was pushed onto whoever reviews. If your approval time also grew, the team is merging more and understanding less — exactly the combination in the 302k-commit study.
4.2 Does verification have an owner, or is it hope?
In all three cases, someone is responsible for saying "this can go in": review with a rule, tests in the pipeline, an explicit acceptance criterion. If at your company the answer is "each dev reviews what the AI generates", you don't have a process — you have an implicit policy of trusting a tool that 46% of developers say they don't trust. Tool configuration versioned in the repository and checked in CI is the first step, described in Your AI usage policy doesn't prevent the leak.
4.3 Can the team explain the code it merged last week?
It's the comprehension-debt test, and you can run it literally: pick an AI-assisted PR from last week and ask the author to explain one decision in the code. If the answer is "that's what the tool generated", you're accumulating the 22.7% — the problem nobody fixes because nobody understands it.
The opposite also holds
If your team is small, PRs are the same size and review is current, don't create a new process because of this post. Industrialized verification solves a volume problem; without volume, it's bureaucracy looking for a problem.
5. The step before any tool
Notice the three questions are about the flow, not the model. That's why switching assistants doesn't change the outcome for anyone who answered all three badly — and why the cases work with different tools.
The AI strategy that works moves together with the development culture the team already has: the review ritual, the pipeline, the criterion for what gets in. The right tool chosen before that becomes shelfware. The right flow finds the tool on its own.
How we help teams adopt AI
We work with engineering teams putting AI into the real development workflow — not as an experiment, but as an actual part of how the team ships.
That means choosing the right tools for the team's context, configuring them in a way that makes sense for the company's data policy, and making sure developers know how to use them in a way that genuinely increases productivity instead of adding friction.
If you have a development team and you're trying to put AI to work seriously, get in touch.