I Handed AI my Raw Video, Audio and GPS from a Round of Golf and This Happened

What We Just Did

My son Ben and I played a round at Sage Meadows Country Club in Jonesboro, Arkansas.

The goal: break 69 in a mulligan scramble and a fifth of Jack Daniel’s Sweet Mash Whiskey. We filmed it on three cameras, wore two wireless mics, and my clubs had Arccos sensors on them.

Then I did something I’ve never done: I didn’t open a video editor. I handed everything to Claude Code, the AI agent I work with every day, and told him what I wanted in plain English.

Here’s what he did, what he got wrong, and what I learned.

Step 1: Transcribing the audio

The first thing I saw him do was transcribe both mics right here on my own computer, for free. He then was able to sync that transcription back to the audio and I knew I could say, “Jump to where I said, ‘that’s down the right.'” It was so easy to guide him based on what was being said at any point.

Step 2: Ask Arccos what happened

Arccos records every swing. So I figured the shot map was the easy part, and getting the data was easy for golfClaude.

Arccos stores where a ball started and where it is hit the next time, and sees that distance as the travel distance for that strike. One limitation is that its clock only knows when my sensors fired. In a scramble, Ben hits his own clubs, so only my shots were known, but that was the ball we played, so it worked.

golfClaude figured this out by comparing Arccos to the audio. Every club strike is a loud spike on the hitter’s own mic. Ben’s drive: 3:47:20.86 pm, +25.7 dB on his mic. Arccos said Brian’s hit was at 3:46:49.

One clock for every recording, and where Arccos got the times wrong

Step 3: Find the shots Arccos never saw

On the first hole, my drive went right. We didn’t play it, so Arccos never recorded where it landed.

golfClaude’s answer: our golf cart camera has GPS. We stopped the cart to pick up my ball. That stop is where my ball was: 160 yards out, 53 yards right of Ben’s.

I told him:

"You did not put my first shot on the Arccos map in the right spot… you have the GPS location… draw it there."

The map inset: Ben's drive from Arccos, mine from the cart's GPS

Later, I had to remind him how a scramble works:

"The second SHOT is from Ben’s location, not my ball location."

He fixed the line in one pass, and he saved that rule so every future hole starts from the ball we chose.

Step 4: Put everything on one clock

Phones keep network time. The mics and the GoPro keep their own clocks, and they drift. golfClaude lined them up by listening: he found the same sounds in every recording and matched them. He called it done. I watched it and it was still half a second to two seconds off.

So I gave him the idea:

"The golf shot is a distinct sound. That should allow you to sync all cameras with that sound."

It worked on the first try. Every GoPro clip agreed to within 30 milliseconds. Now every word, every swing, and every map line sits on one real clock.

The first try

Here’s what he built within hours of getting home from the course. Every camera at once, in a grid:

First try: a four-pane grid of every camera (10 seconds)

Watch the 10-second clip

This was perfect for me to watch the total 18 holes and see where we forgot the handheld camera, where we had bad angles and which of our banter is the most important.

Then I said:

"We are not doing multi-frame any more… this is a full screen video with the pic-in-pic at times."

How he got better, one note at a time

The early map was a flat line laid on its side:

Early version: a sideways map and a straight line

"The map needs to be vertical not horizontal."

"There is no flight of the ball, the line just appears. I’m hoping for a ball flying thru the air feeling."

So the lines now bow and grow from the moment of impact, and the ball rides the tip.

Then I noticed the picture looked soft:

"Has this video been ultra compressed?"

It had. golfClaude had been working from small 720p copies to save time. He rebuilt everything from the 4K camera files:

Same frame: 720p copy vs the 4K original

I’ve seen Adobe Premiere plugins for podcasters that let you edit the video by editing the text. golfClaude gave me the transcription of a talking scene and I was able to just give him the sections I wanted to hear and he pieced it together perfectly.

The scorebox was pretty but unreadable:

"The 3 titles in the top graphic ‘Hole’ ‘Par’ ‘Shot’ are really hard to see."

Scorebox before and after

For the first 25 versions of his edits, I always missed seeing the score change from E to -2. And the eagle moment needed drama:

"Some cool shaking effect of the graphic that when it settles the number THEN changes."

It grabs your attention now!

Ben is a musician, so his song plays over the cart ride, with a QR code to hear the rest, my part in that was “Start at that R2D2 sound in the intro when the cart starts, and then drop the volume to 7% until the title slide hits and go back to 35%. Oh, and put a qr code to his SoundCloud cut of that song."

QR code to Ben's song on SoundCloud

The detour

For days we chased a crazy idea: replace Ben and me with cartoon versions of ourselves. It almost worked:

Real footage (left) and the AI cartoon version (right)

Faces drifted. Lips didn’t match the words. Every attempt burned real credits and still missed. Higgsfield.ai is amazing, but I gave up and pushed in another direction:

"I really thought this would work but the tech just isn’t there yet."

We shelved it. The real footage is better this way.

The same ten seconds, 31 versions later

Final version: the same moment as the first try

Watch the 10-second clip

Then the end card, which he built with a live screenshot of this website:

End card: Edited Entirely by AI

"That title screen at the end ROOOOKKKKKKKSSSSSSSS"

What I learned

golfClaude did the hard parts I’d never have done. Syncing five recordings by sound. Pulling GPS off the cart. Hearing club strikes in the audio. I wouldn’t have known where to start.

I did the part AI can’t: taste. Thirty-one versions of hole 1, and almost every change was me saying “that’s not quite it.” golfClaude never got tired of it. But I’m hoping that number decreases quickly.

Plain English is the interface. I didn’t have to name timecode or a filter. I said “that black screen needs to end shortly after the music drops off”, and he measured the song and did it.

He remembers. When I corrected him, he didn’t just fix the one spot. He wrote the rule down so the next hole starts right.

This was hole 1. There are seventeen more.

Edited entirely by AI. Directed by me. Feature Image by AgenticBen.

A Claude Code session where Brian gives golfClaude edit notes on the video
Example of an edit session with golfClaude

Leave a Comment