Poor detection with my test sample

[MathieuDuponchelle]

Hey, I'm studying existing ball detection solutions, came across your repository and gave it a test with a sample I picked from youtube, it is a wide shot, filming the entire pitch with a pretty shallow angle, and the ball is very rarely detected accurately. I'm merely posting as feedback, but I'll be curious if you have an idea what makes this sort of footage a bad fit. The sample is up at mathieudu.com/misc/clay.avi :)

[RvI101]

I'll check it out, on first glance, this is quite a different viewpoint from what it was trained on. Is the ball not detected when it gets close to the camera, or is it inaccurate in all areas of the pitch?

On Sat, Oct 31, 2020 at 8:49 PM Mathieu Duponchelle < notifications@github.com> wrote:

Hey, I'm studying existing ball detection solutions, came across your repository and gave it a test with a sample I picked from youtube, it is a wide shot, filming the entire pitch with a pretty shallow angle, and the ball is very rarely detected accurately. I'm merely posting as feedback, but I'll be curious if you have an idea what makes this sort of footage a bad fit. The sample is up at mathieudu.com/misc/clay.avi :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RvI101/DeepBall-Keras/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEFKX43PTTQ67VT4OQAWKLSNSV7ZANCNFSM4TGGI2EA .

-- Regards, Arvind

[RvI101]

Ok, I ran it myself. It's getting confused by the sky and the clouds. If you lower THRESHOLD_VAL in DeepBall.ipynb to 0.999, you get some hits when the ball is closer to the camera, but otherwise it predicts a spot in the foliage peeking into the clouds as the most likely position of the ball. I think it needs to be retrained on these sort of samples to get it to work, maybe a bit of transfer learning might work.

[MathieuDuponchelle]

Hey, thanks for taking a look so quickly!

I had also lowered the threshold and observed the exact same behaviour indeed (occasional accurate detection when the ball is close, somewhere in the foliage the rest of the time). I only read the introduction of your paper tbh, and it did mention background subtraction, but is this something implemented by this solution, or a preliminary step that should be performed prior to running the detection?

fwiw, the use case I hope to implement is as follows:

• put a stereoscopic, very high resolution camera up a pole and capture our matches from such a static angle (though I could make sure to capture as little background as possible)

• detect the ball as often as possible in the footage

• pan and zoom inside the footage, trying to interpolate and get as smooth a movement as possible for the "virtual camera"

I'll crop the background out manually with that clip, trivial as the camera is static and report back with the results :)

Note that all this is very theoretical for now, as my country is on lockdown :(

[MathieuDuponchelle]

I'm afraid the results aren't better with the background cropped out. It looks to me like the detection is very size-dependent, which makes it ill suited for wide angle shots, it's really only a guess ofc, I won't pretend to understand the implementation :) Do you know of other ball-tracking approaches worthy of note? I had a brief go at neural networks some time ago, and I remember using LSTMs, which took advantage of the information from previous samples at the cost of some latency, it seems to me like that could be helpful for such a use case.

[RvI101]

Yes, the model was trained on long-shot match footage, where the ball is very small. So the size discrepancy is probably confusing it. You can try retraining/transfer learning if you have annotated footage in your format. I haven't used LSTMs ever, sorry, can't really help there. But your use case is very exciting and I'm sure there's a way to do it.

[MathieuDuponchelle]

Yes, the model was trained on long-shot match footage, where the ball is very small.

Hrm, given the accurate matches occur when the ball is close, I guess when it is far you could consider it smaller than very small then :) In any case, size sensitiveness is an issue, but I suppose retraining with data more similar to the end goal could help indeed. Annotating data sets however is a bit boring :D

I haven't used LSTMs ever, sorry, can't really help there.

ack, I'm just mentioning that because it seems like it would match the use case well intuitively, I don't have anything to back that up though :)

But your use case is very exciting and I'm sure there's a way to do it.

I also think it is a fun project, but I don't necessarily have too much free time to sink into it, I'll continue looking for ready-made solutions. I don't mind the issue being closed, but if you plan on improving this model / trying out different approaches and remember to ping me when you have something new to test that would be very nice :)