ML edge processing for HDC-S

[punov]

• ✅ Get readable detections out of VPUX-optimised Blob execution
• ✅ Tune for better detection quality: pick model, fix script
  • Turned out detections are much (MUCH) better if we resize images not to squares for running inference, but making a square keeping the original ratio of imagery by adding paddings. I'm seeing x2 better results out of this simple fix
• ✅ Adjust old ML script for running NEW inference
• ✅ Load 2 models to switch between, for improving the performance on high-speed collections
• ✅ ODC API: Instead of packing FrameKM immediately, wait till ML processed, only then pack
• ✅ ODC API: Make sure packaging doesn't stuck if ML fails / script crashes
• ✅ ODC API: revisit ML instrumentation, make sure all the telemetry & metadata is collected
• ✅ Prepare build for Masaya's drive testing
• ✅ Validate blurred-on-the-edge imagery on server side ~✅ Revisit kill switch flags: worked bad previous September. We should be able to shutdown it faster when needed~ (Separate ticket)

[jayalberts]

Evan to sync with Alexei this afternoon. Hoping for drive test tomorrow.

[punov]

WIP branch: https://github.com/Hivemapper/odc-api/pull/149 ✅ Load 2 models to switch between, for improving the performance on high-speed collections ✅ ODC API: revisit ML instrumentation, make sure all the telemetry & metadata is collected Fixing E2E defects to make the process stable

[jayalberts]

aiming for Monday drive test, intensive highway drive required.

polishing defects today, sometimes getting crashes. tooling to see if camera is alive.

[jayalberts]

additional optimizations made, switching things from CPU to VPU.

[Masaya-RT]

Ran tests on 4.0.10 and 4.0.11:

• 4.0.10 had issues with gaps, Alexei to fix
• 4.0.11 fixed these gaps, Alexei & Jay deployed to CTP