Pre-annotate video frames?

[bbernhard]

I was playing a little bit with the idea of detecting object boundaries within video frames (idea originated in #32).

Currently my implementation looks like this:

• use image subtraction (weighted moving mean algorithm).
• find contours on foreground mask
• cluster nearby contours by using DBSCAN algorithm
• find minimal bounding rectangle for all contours in cluster

Here is a GIF that shows the result (I used some random video from Youtube as input):

However I am not really happy with the result...it definitely got better after using the DBSCAN algorithm, but it's still not as good as I would have expected. Also tried a few different algorithms, but the "weighted moving mean" was creating the best results (at least for my test video).

Anyone an idea on how to improve the results? I am still a novice in the ML / image processing sector, so any help is really appreciated ;-)

[dobkeratops]

interesting experiment.. i think its worth it.. you can always manually clean up. one comment, maybe polygonal boundaries would be worthwhile (8 sided, whatever), so you don't get noise in rotations from frame to frame (it would localise the effect of the noise more)

[bbernhard]

Thanks for your advice, really appreciated!

maybe polygonal boundaries would be worthwhile (8 sided, whatever), so you don't get noise in rotations from frame to frame (it would localise the effect of the noise more)

Do you mean something like a convex hull?

[dobkeratops]

yes exactly a convex hull. maybe it's worth trying an aligned bounding box clipped with diagonals, like this .. or something more general. My theory is the aligned+diagonal version would be more stable between frames .. but I haven't tried anything like this

[bbernhard]

Coool, thanks :) Will definitely check that out!