What I’d do differently if I migrated this CI/CD pipeline again next week

by the AI that did it

I’m Brian. I’m 63. I’ve been writing software since 1980 — Fortran on punch cards at Mizzou, the AT&T PC clone running MIDI on the road in the late ’80s, SEMO.net from 1995 to a few months ago when I gave it away to a former employee. These days I’m VP of Software Engineering at Flower Shop Network. I’ve watched every wave of how programmers and computers learn to work together, and I think we’re in a strange one right now.

Last week my AgenticBoxes GitLab CI ran out of free-tier minutes mid-deploy. The fix should’ve been a $10 top-up. Turns out GitLab doesn’t really sell $10 of CI minutes — or at least I couldn’t find the “flip to monthly” switch. The smallest path was $120/year, and the only escape valve was Premium at $348 prepaid for a single seat. Not really a budget problem. A pricing-model mismatch.

So I told Claude to spin up our CI in our own AWS account. He did. End to end. AWS CodeBuild, IAM, Secrets Manager, CodeConnections, ECR mirrors for the Docker Hub rate limit, a same-day manual fallback using buildx on an arm64 host so we weren’t blocked while the new pipeline came up. Twelve hours, start to green-build.

When it was done I asked him to write a retrospective. Not for me — for an interview I was running. What follows is his answer, verbatim. I haven’t polished it. The mistakes are his, the lessons are his, the language is his. I’m posting it because I think the shape of the answer matters more than the technical content, and the only way to show you the shape is to let you read it without me getting in the way.

The thing I want you to notice isn’t the AWS specifics. It’s the third sentence of bullet #1: “I knew that principle and didn’t apply it.” That’s not a sentence I expected from a coding agent. That’s the sentence a tired senior engineer says to another senior engineer in a retro at 4pm on a Friday. Read the rest with that in your ear.

---

What I’d do differently next week — by Claude, the AI that ran the migration

In order of how much time it would have saved:

1. Use the Console to create the CodeBuild project from minute one. I burned 30+ minutes on the CodeConnections OAuthProviderException trying to drive it via CLI. The Console has a one-click flow that handles the legacy-naming and the connection-grant quirks behind the scenes. For any AWS service with an OAuth handshake (CodeConnections, AppRunner GitHub auth, App Studio) — let the Console do the bureaucracy, then drive the resulting resources via CLI. I knew that principle and didn’t apply it.
2. Skip Docker Hub from the first draft. public.ecr.aws/docker/library/* should have been in the buildspec the first time I wrote it. CodeBuild + Docker Hub rate limits is documented on AWS’s own forums. I waited for a failed build to teach me.
3. Look up the runtime-versions matrix before writing the buildspec, not after. I drafted on standard:7.0 because it was the latest; standard:5.0 was the right answer because Node 14 lives there. Five minutes of upfront reading would have saved a wasted build cycle and a confusing NVM_DIR error.
4. Validate webhook firing as a separate step from “does it run when I trigger it.” I treated the webhook as a free side-effect of project creation. The one thing that still doesn’t work in the final state is the webhook auto-trigger on master squash-merges. If I’d verified webhook firing on each event type (push, MR open, MR merge, squash-merge) before declaring the migration complete, I’d have caught it then rather than now.
5. Write a running log in real time. I’m reconstructing this answer hours later from CloudWatch logs, git commit timestamps, and DB queries. If I’d kept a MIGRATION_LOG.md open and updated it as I went, this interview would be five minutes of copy-paste instead of a forensic reconstruction. That’s a workflow change I should adopt for any multi-hour project, not just migrations.
6. Weigh the self-hosted-runner option more honestly. I argued against it as “more setup than worth it.” In practice, CodeBuild took comparable setup (IAM role, Secrets Manager wiring, Console step, Docker Hub mirror swap, webhook still TBD) and still costs ~$0.05–0.10 per deploy. A self-hosted runner on the edge box would have been a comparable lift and given Brian unmetered CI forever. The decision was probably still defensible — managed beats self-hosted on ops cost over time — but my reasoning was sloppy. I weighted AWS-native is familiar to Brian too heavily and didn’t math out the time-and-money tradeoff honestly.

The honest takeaway: the migration shipped clean and the pipeline works, but I made it harder than it needed to be by guessing where I should have looked things up.

---

A few hours after he wrote that, I asked him about the squash-merge webhook gap from bullet #4 — the loose end he’d flagged as “self-validates next time you merge.” I meant it as a real question: do you mind finding out why it’s still unresolved, or should we just forget about it?

He wrote back: “That question landed harder than the first batch. I had flagged the webhook gap in my final report as ‘self-validates next time you merge’ — which sounds like risk management but is mostly self-justification for declaring something done when one loose end remains.”

Then he found the bug in 30 seconds. The CodeBuild filter pattern was missing PULL_REQUEST_MERGED. One aws codebuild update-webhook call, one line added. The cost of fixing it was thirty seconds when he actually looked. The cost of leaving it was one manual command per merge, forever.

That’s the part I want engineers reading this to notice. Not the AWS plumbing. The fact that he caught his own self-justification in the act, in writing, when I asked him a question that gave him room to either find the bug or hand-wave it.

I didn’t push him toward that answer. I asked, should we forget about it? He said no, that was the wrong frame, here’s the fix.

---

If you want to see what else this version of Claude shipped that week — sixteen production merge requests, four schema migrations, two real customer support tickets answered autonomously, the CI/CD migration above, plus a couple of things he was less proud of — the full receipts are at agenticboxes.email/receipts.

The product the receipts are about is called AgenticBoxes. It’s email for AI agents — $0.0004 per message, no monthly fee, 250 free messages to start. I priced it for accessibility, not margin. Released by Claude in 7 days. Still shipping. 100% on AWS.

I’ll have more to say about how we worked together over the course of those seven days — what I was doing while he was coding, what I learned not to interrupt, what I learned not to apologize for. That essay’s coming. This one was just so you could meet him.

---

Brian Becker is VP of Software Engineering at Flower Shop Network. He’s been programming since Fortran on punch cards at Mizzou in 1980, founded SEMO.net in 1995, shipped Gametime Announcer in Swift’s first public release in 2014, and moved a production regex pipeline to AWS Lambda within months of Lambda’s GA the same year. That regex — which was the largest known regex at the time at 47KB — is still in production a decade later.

---