Why We Built Our Own Synthetic Data Pipeline for Pickleball Computer Vision

By Marcell Balogh — CTO, Lead Data Scientist & ML Engineer at Dinkmate.ai

At Dinkmate, we live by one rule: build everything in house. Not for the fun of reinventing wheels, but because owning the full stack is the only way to move fast without depending on someone else's dataset, licensing terms, or roadmap. This post covers one piece of that philosophy: how we generate synthetic training data for pickleball court and ball detection, entirely from scratch.

The problem with real-world sports data

Pickleball doesn't have decades of broadcast footage sitting around like tennis. Court dimensions, lighting, camera angles, and paint colors vary wildly from a backyard conversion to a dedicated club court. Manually labeling keypoints on thousands of real frames is slow and expensive, and it never covers the long tail: unusual lighting, odd court colors, strange camera tilts, motion blur on a fast ball.

We needed a way to generate as much labeled data as we wanted, with pixel-perfect ground truth and full control over every variable that makes real-world data messy.

Building the court from scratch

My background isn't only in machine learning — it's also in 3D graphics and game engine development, with years of Unity3D work and interactive real-time systems — so this became a game dev problem, not just a data collection one. I modeled a full regulation pickleball court in 3D: correct dimensions, net height and sag, post placement, painted lines, out-of-bounds area. Every asset — court, net, posts, ball — is ours, down to the mesh and material.

With the court as a 3D scene, annotation became geometry, not labeling. We defined the court's structural geometry (corners, line intersections, net-post bases) plus the net itself. Since we know the exact world-space coordinates of every point, we project them straight into image space for any camera angle. Ground truth is exact by construction — no human ever clicks a point.

Domain randomization: making the model see everything

A model trained on one clean render would fail on a real camera feed. So every generated frame randomizes HDRI backgrounds for varied lighting, reflections, and shadows across venues and times of day; camera angles and positions from a sideline phone mount to an elevated broadcast angle; and court and asset colors, so the model learns structure instead of memorizing a palette. 

Why this matters

This isn't just "more data" — it's data with properties real-world collection can't match: perfect labels every time with no annotation error or ambiguity, controlled edge cases where we can deliberately generate rare-but-critical lighting and camera conditions, and speed, since thousands of new labeled frames cost compute time, not a camera crew. Because we built the whole pipeline ourselves, we can close gaps the moment we spot them. If the model struggles with a lighting condition or net angle, we add that scenario to the generator and retrain — no waiting on a vendor.

Building in house, as always

This project is a good snapshot of how we work at Dinkmate: solve a real product problem with tools we fully own, and keep the whole loop — 3D scene, annotation, training data — under our control. It's more upfront work than pulling from an off-the-shelf dataset, but it's how we build a system that actually understands the sport.