How we actually use AI in our companies
and why we never leave it unsupervised
There’s a lot of noise around AI right now. Depending on which corner of the internet you visit, it’s either about to replace every software developer on the planet, or it’s a glorified autocomplete that produces nothing but bugs. We’ve been using these tools daily across our companies for a couple of years now, and neither camp matches what we actually see. The reality is messier, and a lot more interesting.
Jack Conte, the CEO of Patreon, recently gave a talk called The Future of Creativity on the Web in a World with AI, and it has something you rarely find in the public debate: nuance. He admits he feels conflicted about the whole thing, outraged and fascinated at the same time, and points out that the loudest voices on social media capture very little of what’s really going on. On one point he’s firm, and we agree with him: this is still the very beginning. Most of what these tools will eventually be good for hasn’t arrived yet.
Where AI earns its keep
Code review. An AI assistant reviewing a pull request doesn’t get tired, doesn’t skim, and has no problem telling a colleague they forgot to handle a null case. It catches the boring, mechanical stuff (inconsistent naming, a missed edge case, a forgotten error path, the odd off-by-one) before a human reviewer even opens the diff. The human review that follows can then spend its energy on things a model can’t judge: architecture, intent, and whether the change makes sense for the business in the first place.
Writing tests. Nobody loves writing tests. AI doesn’t seem to mind. Give it a function and you get back a test suite covering the happy path, the edge cases, and a few failure scenarios nobody would have bothered typing out on a Friday afternoon. Test coverage across our projects has gone up, and not because anyone mandated it. Writing a decent test simply got cheap, so people write more of them.
Day-to-day programming assistance. Scaffolding a new module, turning an idea into a first working draft, explaining an unfamiliar codebase, writing the tedious glue code between two APIs, drafting documentation. For this kind of work AI behaves like a very fast, very well-read junior colleague. It won’t make the hard decisions for you, but it clears away a surprising amount of friction around them.
The part everyone skips: supervision
Now for the caveat. We don’t treat it as a footnote; for us it’s the core of the whole approach: none of this works unattended.
AI-generated code looks confident, and that’s exactly the problem. It compiles, it reads well, it’s formatted beautifully. Sometimes it’s also wrong in a quiet, plausible way. It will invent a library function that doesn’t exist, misunderstand a business rule it was never told about, or “fix” a bug by removing the behaviour that was the whole point. Stop reading the output because “the AI is usually right”, and sooner or later one of those mistakes ends up in production.
So we treat AI the way you’d treat a very new team member. Everything it produces gets reviewed by a person who understands the context. It never merges to the main branch on its own, and it never touches production data. Whoever accepts an AI-generated change owns that change, as if they had typed every character themselves. Accountability doesn’t get delegated to a model.
The teams that get the most out of AI tend to be the ones that supervise it most carefully, because careful supervision is what lets them hand it bigger and bigger tasks.
Slow down to speed up
We’re clearly not the only ones landing on this. Gergely Orosz, who writes The Pragmatic Engineer, recently gave a keynote with a title we could have come up with ourselves: Slow down to speed up: AI and software engineering.
With agents in the loop, teams ship far more code than before: more pull requests, and much bigger ones. What doesn’t grow along with it is attention. A rising share of changes gets approved without a human actually reading them. Orosz points out that individual productivity is up while team productivity has barely moved, and that gap should worry anyone running an engineering team.
The bottleneck has moved. When code is cheap to produce, the expensive part is deciding what to build, reviewing what came out, and telling the difference between good and merely plausible. Orosz calls that skill “taste”, which feels right to us. The teams doing well with AI aren’t the ones generating the most code. They’re the ones that put real effort into planning before handing work to agents, and into review and quality control afterwards. Slowing down at those two points is what lets everything in between run fast.
Still the beginning
Coming back to Conte’s point, what strikes us most is how early all of this still is. Today’s tools are the worst versions of them we’ll ever use. Two years ago AI could complete a line of code; today it can review a pull request, draft a test suite, and reason about a refactoring spread across a dozen files. Looking at that trajectory, it’s hard to argue we’ve already seen what these tools are for.
One lesson, though, will probably hold for a long time: AI amplifies the people who use it, in both directions. In the hands of an engaged developer with good judgment, it’s a real force multiplier. Left alone, it’s a very fast way of producing very convincing mistakes.
The potential is ahead of us. The responsibility stays where it has always been – with us.
This is also how we run the software subsidiaries we build for our clients in Poland. If you are curious what that looks like in practice, have a look at how working with us goes, in four steps.