Data vs. Architecture: What We Learned Building Ball Tracking for Pickleball

By Marcell Balogh, CTO / Lead Data Scientist & ML Engineer, Dinkmate.ai

There's a comfortable story going around ML circles right now: get enough data, point a solid off-the-shelf model at it, and the problem basically solves itself. Architecture matters less than it used to — data is king, or so the story goes.

We believed a version of that too. Turns out the devil is in the details. Without a real, purpose-built architecture, no amount of data saves you — the ball's just too small, too fast, and too easy to lose against a cluttered court. We still think data matters enormously. We just learned the hard way that it's not everything.

The problem nobody warns you about

A pickleball is small, it's light, and it moves fast — fast enough that on a lot of frames it isn't really a ball anymore, it's a blur, a smear of a few pixels, sometimes half-hidden behind a paddle or a player's arm. Standard object detectors, including strong general-purpose YOLO-based ball models, are built and tuned around objects that look like objects: a clear shape, a reasonable number of pixels, consistent edges. A blurry, half-occluded pickleball constantly breaks those assumptions.

So we did what the current wisdom suggests: we collected data. A lot of it. Thousands of labeled frames from real games, covering different courts, lighting, camera angles, ball speeds. And the off-the-shelf models still missed the ball in exactly the moments that mattered most.

More data wasn't the fix

This was the uncomfortable part. We kept assuming the next batch of data would close the gap. So we did what everyone does — more footage, then heavy augmentation. It didn't matter. We 10x'd the dataset and the model still lost the ball in the exact same spots.

That's when it clicked: this wasn't a data problem, it was an architecture problem. A model built to find clean, well-defined objects doesn't get better at finding blurry ones just because you show it more blurry examples, real or synthetic. The bottleneck wasn't the data — it was the model's baked-in assumptions about what "the ball" looks like.

So we built our own detector from scratch, designed around the actual failure case: a small, fast, frequently-blurred object, rather than a generic small-object detector borrowed from another domain. That meant rethinking how the model represents motion blur, how it handles partial occlusion, and how confident it's allowed to be about a ball-shaped smear versus an actual ball.

The real lesson

Data and architecture aren't competing priorities — they're both necessary, and neither substitutes for the other. Good data with the wrong architecture gets you a model that's very well-trained at failing. The right architecture with no data gets you nothing at all. We needed both: our own pipeline collecting real game footage, and our own model built specifically for what a pickleball actually looks like.

It's tempting to treat "get more data" as a universal fix, because it's the answer that doesn't require touching the hard part. Sometimes it really is the fix. But sometimes the technology itself is the gap, and no amount of data closes it until you build the thing that was actually missing.