When to Use AI-Generated Code
Answering an Increasingly Important Question
If you haven’t noticed, AI is becoming increasingly proficient at writing code. The question of whether and when to use AI-generated code over bespoke, human-written code is one that I often ask myself when working on projects, and that software managers and developers, in general, will have to grapple with as these AI models become more deeply embedded into the software development lifecycle.
When considering this question, three possible answers arise: always, sometimes, and never. However, in my view, the first and last options, always or never, shouldn’t be taken seriously. The only serious answer is sometimes—but when? Even so, there are some misguided or foolhardy types who may venture to say always, and for the sake of completeness, the stoical “never” position also deserves addressing. So let’s work through these different stances.
Always
The “always” camp is insidious, as it is both absurd when examined closely and tempting for those in power who wish to cut costs by eliminating programmers. On the surface, there is a superficial appeal to the idea that we should just stop coding and prompt for code or fully automate humans out of the loop. Coding kind of sucks; it’s frustrating, expensive, and challenging. Bite into this glossy apple, however, and its core will prove rotten.
I foresee a genuine risk in a few years of the rise of zombie apps, apps that are thoughtlessly prompted into being by people with no respect for or understanding of the software engineering discipline, or worse, automatically generated with zero human supervision. These zombie apps will be slapped together, unreviewed, nebulous heaps of code destined for malfunction and neglect. Already, I am seeing it happen: a flood of dilettantes and prompt-kiddies with no interest in following time-honored development practices, thoughtlessly copy-pasting AI-generated code and pushing apps that are fated for nothing but doom. These zombie apps will suffer from poor maintenance, low performance, and will be riddled with security vulnerabilities and anti-patterns. They won’t scale either, so most serious people won’t even bother with them.
Such zombie apps may work in a nominal sense, but they will lurch on in a kind of half-life, only barely usable, and essentially just waiting for something to arise in the greater tech ecosystem that puts them under for good.
At first, the ability for these AI models to confidently spit out code is seductive. The upfront time and cost savings are hard to argue against. But over time, it will prove a false promise, because code that nobody thought through is code that’s just begging to cause trouble.
Another genuine danger of always opting for the prompt is skill atrophy and knowledge loss among developers. Something about writing code sharpens the brain. Losing that intimate knowledge and practice could cause a regression in expertise and an eventual decline in standards, innovation, and progress in software development. Models trained solely on future AI-generated code will also decrease in quality over time, as they lose access to human variation and diversity of processes, potentially nuking the entire training pipeline that model bootstrapping depends on.
And in any case, the more knowledgeable one is about software development, the better one can prompt for code—so the prompt engineers that will replace software engineers are the same people they would replace. Prompt engineering in a software context is just a different technique for software engineering. It’s another means to an end. The more you develop a prompt, the more it begins to resemble plain old programming with a lot of annotation. Any prompter worth their salt just ends up coding by one step of removal.
What if these models become so advanced that they surpass human abilities in generating code? I’ll believe it when I see it. These models certainly won’t be finding superhuman code samples in their training data. So whatever is accomplished to make them better at creating code than the best humans has to be something exceedingly clever.
I’ve watched the coding abilities of generative AI evolve from day 1, and while they have consistently gotten better, there is still something fundamentally lacking—judgement. They’re not really capable of viewing code in a complex, nuanced context, and there are many subtle decisions that developers make which never seem to enter their process.
Only people who don’t understand the intricacies of software engineering would think that we should or could replace it entirely with AI bots.
Never
The “never” faction deserves at least an honorary mention. I don’t know of any developers who, pure of spirit, swear off AI-generated code completely. These new-age John Henrys, whoever they are, prefer to race against the machine using unaided muscle alone, either out of stubbornness, inertia, pride, or denial. These developers likely scorn the new wave of dabblers entering the field who vastly underestimate the complexity of coding due to the ease of prompting AI for something vaguely usable. Most of these never-AI types are also probably acutely aware of the problems that uncritically prompting for code or automating code generation can cause.
However, the position that staunchly rejects using ANY AI-generated code or AI coding assistance generally is beginning to look woefully behind the times. The efficiency gains of incorporating AI into software development are simply undeniable. The numbers game isn’t looking good. Those who can use AI tools to produce usable, decent, if not master-crafted work faster than those who take twice or thrice as long to do things right will probably get further ahead in this kind of fast-paced, results-oriented industry. Old-school programmers need to get with the times and recognize that they are in a field that is always accelerating and changing, and that often chances upon objective qualitative improvements to processes which then ruthlessly override older practices to become the new norm. Denialism or refusal to change is a recipe for getting left behind.
Sometimes—but when?
This brings us to the only position worth serious consideration. Yes, you should use AI-generated code, but only when it’s the right call. The question then becomes: when is it the right call?
In reality, AI’s code generation is an evolution of what software developers have always done—borrowing and integrating code from various sources. It’s just a more efficient method than searching the internet for code snippets, fragments, or boilerplates and snapping them together and paving over the cracks so you have something different. AI code generation is just a way to get a more precise, tailored result from a resource faster. It’s rare in a working context for software developers to start projects entirely from scratch without borrowing, forking, or building on top of code someone else made.
What I foresee happening is that forward-thinking software engineers will develop very carefully constructed prompts that will generate boilerplate to kickstart projects. Then they will customize and refine them. They will learn how to play these models like a fiddle, to get very carefully molded results to start off projects. They may even fine-tune or train models to learn how *they* like to write code, so that the model is always subject to their preferences.
He’s a small sampler of other techniques:
•Prompting a bot to do something you’ve already done a million times and can easily vet, to save time.
•Prompting a bot to complete a pattern that you have already established and that it can easily follow.
•Debugging small gotcha-type bugs.
•Adding comments to code (that you fact-check and verify).
•Testing out different approaches before committing to any one of them, just to see a quick mock-up of a particular line of thought.
Finally, the real kicker:
•Iterating and improving code through an in-depth back-and-forth dialogue: This new conversational programming paradigm is experimental and still in the throes of birth. But I anticipate a new generation of coders who are very comfortable interacting with LLMs at a deep level and who constantly integrate them into their development process. They are still developing software, but they have evolved into something greater than the sum of the parts. Both human and machine are writing code, suggesting improvements, questioning and answering each other, in a deep, meandering, spiraling dialectic.
Conversational development is the future if you ask me. These kinds of developers will even end up more knowledgeable and engrossed in the details of software than old-school programmers who swear off the technique, or careless prompters who don’t bother to gain knowledge about code. They will be engaged with LLMs on a fundamental level and have intricate, detailed conversations about code and how it works, while still doing a good deal of regular coding on the side. Conversational development cycles will be complex, multifaceted, and nonlinear, yet they will be well-documented, literate, and traceable, thanks to the rich conversational histories with LLMs.
In a way this blog post was just a means for me to stealthy plug *conversational programing*, a paradigm I aim to be a pioneer of, and have quite more to say about. It feels undeniable to me that LLMs are just evolving, not ending, the software development field. It’s just going to take people a while to understand what’s going on and adapt.
LLMs are going to grow deeply embedded into software development processes in ways that have not yet even been dreamed of. They’re really just another tool at the developer’s disposal, nothing to be afraid of or intimidated by, but capitalized on.