This is excellent!

[ms82119]

Hey @fenaux I just wanted to let you know how much I appreciate you posting this solution to resolving the ball trajectory on the radar view. It's really excellent. I think it's really clever, and really clear how you step through the process in the notebook. And it works great!

I was not familiar with those techniques until you shared them here. Now I'm starting to look into the paths of the players, since they are also susceptible to the drift induced by the noisy keypoints. I'm looking to find a way to smooth those out too. As you can see from the screenshot, the ball bath is very clean (orange), and I feel some of the same techniques can be used to clean up the player paths (blue and pink).

[fenaux]

Hi @ms82119

Thanks for your kind comment. Yes you are right, Kalman smoothing should work for the player's path too. As the player's dynamic is smoother than the ball's one, a model with constant acceleration should work.

But let me share some thoughts here. The "noise" in paths has two components :

• player detection
• homography calculation Let's speak of the last one. So Far, homography is averaged. The drawback of this strategy is that if we have an outlier the error will contaminate the estimation of the contiguous frames. The rotation part of the homography matrix requests estimation of the camera's pan and tilt. Probably estimation of these variables with Kalman smoothing would enable outlier's detection and more robust estimation. But this is a non linear Kalman smoothing so it is a little bit more complicated. Note : a sudden move of ALL players from one frame to the other in the radar view is a strong cue to detect a problem in homography calculation as it seems to be the case here at one end of the paths.

Looking at your picture I have one question : angles in the ball path should indicate an action of a player. Here we have angles and no player's path. Is it because you did not plot all the player's ?

[ms82119]

Yes @fenaux you are right that I'm not plotting all the player paths in that first image, every time the ball takes a sharp movement/angle we would expect to see a player nearby. In fact this is one of the things I'm thinking more about now, as I can see that sometimes with the smoothed ball movement, and the independently smoothed player movement it can lead to some situations where the player and ball don't appear that close to each other in the radar view, as you can see in these three example screenshots. One shows the ball/player paths before I smoothed them, one with just the ball path smoothed (using your process), and the other with the player smoothed too, but either way there is still a fairly big distance between player 17 and the ball on the radar view. There are a few reasons for this.

• As you said, the homography can be off, and in fact around these frames in the screenshots there is a bit of jumping as the camera tilts and pans. This phenomenon should affect the player and ball transformations equally, so I wonder if there's a case for smoothing the homography first, and then doing a new transform on the players and ball to get the new radar positions...
• The bounding boxes for players can sometimes artificially change size if a player moves in front of another, the AI capture will just box the part of the player that is visible, resulting in the bottom of the box moving upwards, and the radar will also reflect this with a jump in position. I wonder if there's an option to have Yolo place the bounding box around the whole player even when obscured...
• Sometimes, like in this example the ball will reach the player higher up the body, this will give the radar the effect of the ball being above the player position... not too sure on how to deal with this case yet.

I'll upload the code I came up with to do the player path smoothing, it seems to work pretty well, but could no doubt be better as it doesn't really do anything explicit with the homography errors. Btw, the video clip I am using for my work is called 08fd33_4.mp4 (i think i got it from some other example on YT or maybe Kaggle).

Would be great to get your thoughts on this.

[fenaux]

Thanks for your contributions. I will try to process this video when it is possible to find time.

For the problem you mention (that will be more important when a player controls the ball with head and if he jumps !!)

• one idea is to create an new class (possibly with the ball detector) as "ball handler" by annotating frames when a player interact with the ball and changing ball coordinates to ball handler coordinates in that case
• in the meantime a simpler way if there is a "significant" change in direction of the ball at frame i then change the vertex coordinates at i with the clothest player coordinates

for partial occlusion I would first see what kalmann smoothing achieve

but probably camera pan and tilt smoothing should be the priority.