frame_memory option

[knrdl]

Please bring back the frame_memory cli parameter. Without this feature the videos contain unblurred frames, so the output is less usable because the blurring is not reliable enough. I see that frame_memory > 0 might not work with batch_size > 1 but for batch_size == 1 it would be great to have this option again.

Thanks for keeping this useful tool in development!

[tfaehse]

Thank you for the feedback :) It's nice to hear the frame_memory flag had (some) friends! You're right, it shouldn't be too hard to add back. Once I've finished the YOLOv8 integration I'll get on it.

[tfaehse]

I've added a global store of detections, so blurring previous detections is trivial again. So far the memory impact is a bit of a grey area though, it shouldn't be high (a detection is fairly small), but I still have to make sure here - otherwise I'd have to keep it small by removing all detections older than the frame memory option again.

A nice side effect is that it's easy and quick to export detections now - I've added a JSON export flag to the CLI in order to experiment a bit, so far it just outputs the "internal" data format (two points, min and max that form a rectangle).

edit: once I've tested this some more I want to add some actual label formats instead of that JSON option before merging

[joshinils]

Will the "new" frame_memory be symmetric around the detected frame? previously it would only blur the next n frames, not the previous n too. however, i'd like to have it blur the previous n and the next n too. I get that this would mean that the detection-process would need to be pre-computed for a bit and somehow detached from the blurring process, but I think that is worth it.

[graebec]

Hi - are there any news on this yet ? I just tested you tool and find its simply great. but no matter which treshhold I choose the videos I tested keep flashing the license plate. If you like I can contribute by sharing my footage with you ( approx 2 TB of 1080 Go-Pro footage)

[joshinils]

@aaronpohle have a look at this branch, @tfaehse started to implement the memory option again; https://github.com/tfaehse/DashcamCleaner/tree/feature/detection-store

[graebec]

@joshinils - Thank you for that. Maybe you can help me out: When I follow your link I find an identical repo to the one I installed. - Sorry I am rather new to github etc. - Does that mean I make a mistake - or is that branch just not ready yet ?

[joshinils]

Git has branches, every repository has commits which make up changes to files, each commit has a parent, which belongs to a branch. In some cases, a commit belongs to a different branch than their parent, thus creating a different branch. If the parent branch then also has more commits later on, the branches diverge, this has happened here.

In this repository, the "main" branch is the official/release branch with the latest version (because of that GitHub does not display its name by default, but it is possible to link to it directly: https://github.com/tfaehse/DashcamCleaner/tree/main/). The "feature/detection-store" branch is different and has a different set of commits (in the end of the history, the beginning is the same) as the main branch, currently two less and one more than main, so the files are different there.

You can look at a visual representation of the branches here: https://github.com/tfaehse/DashcamCleaner/network

[graebec]

@joshinils - Great ! Thank you very much for your detailed answer. That helped a lot (at least in better understanding ..)

The only issue .. I used the "git checkout 0110aac" command now but when I run the main.py script I receive an error message: TypeError: init() missing 2 required positional arguments: 'weights_name' and 'parameters'

I suppose I am doing it wrong - would you (or anyone else :-) ) be able to put me on the right track ?

[tfaehse]

...there's probably some sort of holiday somewhere still going on, right?

I've looked into it, and I've decided to split up the stages then, otherwise managing memory will be too difficult I think. It'll now be

1. Read frames and run detector
2. Run forward/backward tracking
3. Read frames, apply blur, write frames

As for tracking: it's just a very simple 2D tracker, but at worst it performs the same as a frame memory, at best it actually somewhat fills in the gaps where the detector is failing. The advantage is that it's ridiculously fast, it costs virtually no performance at all.I really have to improve the detector itself though, the weights aren't up to par anymore. If you want to try it out, I've pushed (some of) the code for the tracking to this branch, but only in the CLI and without properly exposing the parameters for now, most things are hardcoded. That comes tomorrow. The version on this branch also still writes two videos, one with the blurred objects and one with red, green and blue boxes for detections, forward tracking results and backward tracking results - it's BGR, so blue are detections, green forward and red backward tracking results.

Edit: something changed in yolov8, which causes the boxes to have an offset in case your original video resolution is padded (e.g. for 1920x1080), I'll have to quickly fix that first: https://github.com/ultralytics/ultralytics/issues/6575

[tfaehse]

Integrated in https://github.com/tfaehse/DashcamCleaner/pull/87

I'll run some more tests, then merge.