[feature] store detections and use detections file

[joshinils]

While developing, I run the same file over and over again with different options. I think it would be good if there is an option to export and import the detections. That way I can run different settings fast without needing to wait on detections each time again and again.

If possible this could also be done via some temporary file-name on-the-fly without an option. Such that if the hash of the file, its size, last modified date or absolute path is the same you can append to the detections, and reuse them with different settings in case the process fails or is aborted.

I do not know how the detections are stored before they are made into bounds objects, and i do not know how to serialize them into a file to be read later on. this probably needs some metadata like frame-number and filename etc. for each temp-file.

also there should be some process to get rid of these temp-files, otherwise i worry they might spam someone's hard drive if they automate blurring clips for video production.

[tfaehse]

That's a nice idea (probably ), and it shouldn't be too hard either. As long as you run the same video anyway you don't even have to store indices, you can probably just read them all one by one since frames and detections should be synced up.

Pickle works fine with Detections, something simple like

with open("test.pkl", "wb") as f:
    pickle.dump(new_detections, f)

you could, for example, save one detection result. You can also put them all in a big list and then pickle them, it works fairly effortlessly. I'll look into it in the future!

[joshinils]

today I looked into storing the detection-mask in separate color-channels, and i noticed that when i crank the threshold all the way down to 0 the time per frame (for my CPU etc.) is about 13sec. compared to 1.1sec with a threshold of 0.2.

(worth noting that the threshold has this kind of effect on run-time)

so the threshold should also be stored, or an output-filename and or input-filename needs to be specified for the stored detections. maybe other parameters need to be stored too, but overall this would just be a nice-to-have feature, not something I think many people desperately need right now

[tfaehse]

Detections (tracked boxes) are stored in a fairly simple JSON format here: https://github.com/tfaehse/DashcamCleaner/pull/87

Reading them back in would be fairly simple too, I can add that later. In the end, all detections/tracks are just pandas data frames now.

[joshinils]

Do you also store the settings used to generate the detections? This would make it possible to resume detections if I stop mid-way and thus makes re-runs faster, and depending on which settings are changed, there is no need to re-do the detections at all, and only merging, and blurring is needed.

I'd appreciate an example for the json as reference here.

[tfaehse]

So far only the detections/tracks are stored. An extract of a simple example for a short testing file I'm running:

{
  "38": [
    {
      "x_min": 922,
      "y_min": 657,
      "x_max": 938,
      "y_max": 659,
      "class": "plate"
    },
    {
      "x_min": 406,
      "y_min": 582,
      "x_max": 416,
      "y_max": 601,
      "class": "face"
    }
  ],
  "39": [
    {
      "x_min": 922,
      "y_min": 656,
      "x_max": 938,
      "y_max": 659,
      "class": "plate"
    },
    {
      "x_min": 402,
      "y_min": 581,
      "x_max": 412,
      "y_max": 601,
      "class": "face"
    }
  ],
  "40": [
    {
      "x_min": 921,
      "y_min": 656,
      "x_max": 938,
      "y_max": 659,
      "class": "plate"
    },
    {
      "x_min": 390,
      "y_min": 580,
      "x_max": 401,
      "y_max": 600,
      "class": "face"
    }
  ],
  "41": [
    {
      "x_min": 921,
      "y_min": 656,
      "x_max": 938,
      "y_max": 659,
      "class": "plate"
    },
    {
      "x_min": 389,
      "y_min": 580,
      "x_max": 400,
      "y_max": 600,
      "class": "face"
    }
  ],
  "42": [
    {
      "x_min": 920,
      "y_min": 656,
      "x_max": 938,
      "y_max": 660,
      "class": "plate"
    }
  ],
  "43": [
    {
      "x_min": 920,
      "y_min": 656,
      "x_max": 938,
      "y_max": 660,
      "class": "plate"
    }
  ],
  "44": [
    {
      "x_min": -366,
      "y_min": 583,
      "x_max": -435,
      "y_max": 646,
      "class": "plate"
    },
    {
      "x_min": 920,
      "y_min": 656,
      "x_max": 938,
      "y_max": 660,
      "class": "plate"
    }
  ],
  "45": [
    {
      "x_min": -355,
      "y_min": 584,
      "x_max": -420,
      "y_max": 647,
      "class": "plate"
    },
    {
      "x_min": 919,
      "y_min": 655,
      "x_max": 938,
      "y_max": 660,
      "class": "plate"
    }
  ],
  "46": [
    {
      "x_min": -343,
      "y_min": 586,
      "x_max": -405,
      "y_max": 647,
      "class": "plate"
    },
    {
      "x_min": 919,
      "y_min": 655,
      "x_max": 938,
      "y_max": 660,
      "class": "plate"
    }
  ],
  "47": [
    {
      "x_min": -331,
      "y_min": 588,
      "x_max": -389,
      "y_max": 648,
      "class": "plate"
    },
    {
      "x_min": 918,
      "y_min": 655,
      "x_max": 938,
      "y_max": 660,
      "class": "plate"
    }
  ]
}

[joshinils]

Ok, I think a bit of metadata for the file used can't hurt. And the confidence should definitely be stored too, since it exists.

Thinking of stopping the program mid-execution, when does the JSON get stored to file? Probably best to do that as soon as possible.

[tfaehse]

That's actually a bit tricky, since the detection scores kind of lose their meaning after (this sort of) tracking. Storing the detection results and the tracking results separately wouldn't be an issue at all though, and including the scores there would make total sense.

I'll add this for the next feature then, as follows:

• store raw detection output, including the exact parameters used
• on launch, check if a JSON with the same config exists and skip detection stage