Keeping Up With the Code AI Writes
A retry loop can be easy to read. It catches an error, waits, and tries again. The names are sensible. The function is short. You can follow every line.
Then someone asks what happens if the task is cancelled during that wait.
Does the next attempt still start? Can it repeat a side effect? Who decided that three attempts were acceptable?
You scroll back up.
This is one of the uncomfortable parts of working with AI-generated code. You can recognize what the code does locally without having a good enough model of the system to change it safely. And an agent can keep adding working features while that gap grows.
The useful question is how to keep your understanding moving alongside the implementation.
Readable code is only the beginning
When you implement a feature yourself, you encounter decisions along the way. Where does this state belong? Which failures are recoverable? What should happen when two operations overlap? You try something, discover a constraint, and adjust.
Those decisions help build a mental model. Delegating the implementation can skip that process for you. You receive the finished path through the problem, with fewer reminders of the alternatives and assumptions behind it.
Reading afterward can rebuild that understanding. But a polished explanation is easy to accept without doing much reasoning yourself. A large patch makes this harder, especially when it introduces unfamiliar abstractions or changes several behaviors at once.
This can happen with code written by another person too. AI makes it easier to accumulate changes faster than you can absorb them.
What the research actually supports
The evidence deserves some care here.
In Naser Al Madi’s study of 21 participants, model-generated code was comparable in readability and complexity to code written by human pair programmers. Eye tracking nevertheless showed significantly less visual attention to the generated code. Readability and careful inspection don’t automatically travel together.
A two-phase study by Markus Borg and colleagues involved 151 participants, 95% of them professional developers. New participants extended solutions originally developed with or without AI assistance. The researchers detected no systematic maintainability advantage or disadvantage in those tasks. That result is a reason to resist blanket claims that AI-written code is inherently harder to maintain. It also doesn’t settle what happens as a particular developer delegates more of their own reasoning over months.
Judy Hanwen Shen and Alex Tamkin studied developers learning an unfamiliar asynchronous programming library. AI assistance impaired conceptual understanding, code reading, and debugging on average, without a significant average efficiency gain. Their analysis also identified interaction patterns involving cognitive engagement that preserved learning. This was a learning setting, so it shouldn’t be stretched into a verdict on every use of AI by experienced developers.
Taken together, these studies suggest a useful distinction: the quality of the code and the understanding of the person responsible for it need separate attention. The workflow below is a practical proposal for maintaining that understanding, rather than a scientifically validated recipe.
Work through one behavior at a time
Consider a hypothetical task runner. It retries temporary connection failures. Invalid requests fail immediately. Asking for the result of a completed task doesn’t execute it again.
Now add cancellation during the retry delay.
Before asking an agent to implement it, write down the behavior you want:
- A task cancelled while waiting must not start another attempt
- Cancellation is terminal for that task
- Existing handling of temporary failures and invalid requests stays the same
There’s still a design question: what happens to an attempt already in progress? Decide whether this change covers that case. Cancellation can’t automatically undo an external side effect, so avoid promising more than the system can guarantee.
Then bound the implementation. Use the existing owner of task cancellation. Change the retry path and its tests. If satisfying the behavior requires changing scheduling or persistence, surface that decision before expanding the patch.
The boundary should follow one coherent behavior. A ten-line change can alter an important guarantee; a longer mechanical change may introduce very little new reasoning. Line count alone won’t tell you how much understanding the change requires.
Predict before accepting the explanation
Once the patch exists, try a concrete question before reading the agent’s summary:
“The first attempt fails. The task enters its retry delay. Cancellation arrives. When the delay ends, what happens, and which code guarantees it?”
Make your own prediction. Follow the state from the cancellation request to the retry boundary. Find the check that prevents another attempt, and work out whether it observes the current state or a value captured earlier.
Then compare your explanation with the implementation and the agent’s account. If they disagree, you have a specific gap to investigate.
You can ask the agent to explain unfamiliar code, trace execution, or challenge your prediction. Those are useful ways to learn. The important part is that you eventually produce an explanation you can check against the code, rather than simply recognize one as plausible.
Change the scenario, then verify it
Next, move cancellation to a different point: before the first attempt, during the delay, or after completion. Predict which outcomes should change and which should remain the same.
For the delay case, a deterministic test can pause execution at a controlled wait point, cancel the task, release the wait, and check that no second attempt began. Real-time sleeps make this unnecessarily fragile. A controllable clock or wait primitive lets the test exercise the intended ordering directly.
That check provides evidence for a particular behavior. Understanding comes from connecting the result to the mechanism: where cancellation lives, when the retry path observes it, and why the next attempt is prevented.
You don’t need to manually reimplement every patch to do this. Use your attention where assumptions, ownership, and behavior change. Familiar mechanical work can be delegated more freely. Unfamiliar concurrency rules deserve more involvement.
Leave enough reasoning for the next change
A month later, neither a long chat transcript nor “added cancellation support” will be a particularly useful explanation.
Keep a short record near the implementation or in the change description:
- Decision: cancellation during a retry delay prevents future attempts
- Enforcement: the retry path checks current cancellation state before starting the next attempt
- Evidence: the controlled-delay test cancels during the wait and observes no second attempt
- Limit: this change doesn’t undo side effects from an attempt already started
Use actual symbols and test names in the real record. Update it when the behavior changes. This gives the next reader a route through the code and the reason that route matters.
AI can take over a lot of typing. The person maintaining the system still needs to predict what it will do, locate the decisions that make it behave that way, and judge what a change would disturb.
Before accepting the next substantial patch, try that small exercise: predict one important behavior, change the scenario, and verify your answer. If you can’t explain the result yet, that’s the next piece of work.