Spintip
Automatic tennis highlight generation: object detection segments a match into rallies, and the cut is assembled from play, not from a person scrubbing a timeline.
Built at Tezeract, where I was Lead Engineer. I architected and developed this product personally: the AI pipeline, the backend and the system design.
Architecture Overview
A Flask service wrapping a two-stage video pipeline, with a React client for upload and playback. Stage one is cheap: motion and scene gating over decoded frames to discard the long stretches where nothing is in play. Stage two runs the TensorFlow object detector only on surviving segments, tracking ball and player positions to establish rally boundaries. A final assembly step cuts and concatenates the segments into an output reel.
The Challenges
Problem
The obvious framing is to train a model that recognises exciting moments. That framing is a trap: excitement is not a visual property, there was no labelled dataset of it for amateur tennis, and any model we trained would have been learning one annotator's taste.
Solution
We redefined the target into something objectively detectable. A rally is not a matter of opinion. It starts at a serve and ends when the ball stops being in play, and both are observable from ball position and motion continuity. So the system detects rallies, not highlights, and the reel is simply every rally with the dead time removed. The output is defensible for the same reason it is simpler: nothing in the pipeline is making an aesthetic judgement it cannot justify.
Problem
Object detection on every frame of a two-hour match is enormously wasteful when most of those frames are players walking back to the baseline, and on the hardware available it made processing time comparable to the match length itself.
Solution
A cheap gating stage runs first. Frame differencing and scene-change detection over decoded frames identify candidate active regions at a small fraction of the cost of inference, and only those regions reach the detector. Because the expensive stage sees a fraction of the footage, total processing time became a function of how much tennis was actually played rather than how long the camera ran, which is also the behaviour a user intuitively expects.
Problem
A tennis ball is small, fast and frequently invisible: motion-blurred to a smear at speed, hidden behind a player, or lost against a crowd background. Per-frame detection alone produces a track full of holes, and a naive rally-boundary rule reads every hole as the end of the point.
Solution
We stopped treating detection gaps as evidence. Ball positions feed a tracker that maintains a trajectory estimate across missed frames, and rally termination requires sustained absence plus corroborating signals (players out of position, no motion in the court region) rather than a single frame without a detection. Tuning that hysteresis was most of the accuracy work on the project, and it is the sort of thing that is invisible in a demo and decisive in real footage.