search

PhotonVision / photonvision

PhotonVision is the free, fast, and easy-to-use computer vision solution for the FIRST Robotics Competition.
https://photonvision.org
GNU General Public License v3.0
264 stars 180 forks source link

Properly serialize ML models #1209

Open mcm001 opened 7 months ago

mcm001 commented 7 months ago

Actually use the whole models folder for things. Expose a way to CRUD new model files and change their type in a card in the UI, and allow object detection pipelines to choose between different models.

In the backend, we could store models as binary blobs inside a new sql database, or in the models folder and keep a list of (model path, parameters) in the main config database. Or something new I haven't thought of

srimanachanta commented 7 months ago

How are models structures? Is there any performance/quality loss if we blob them into the database? Implementing this into the UI should be fairly trivial (don't quote me)

mcm001 commented 7 months ago

They're just binary blobs of (sometimes several MB of) data. No quality loss in a database, just the tradeoff of increased filesystem robustness vs slowness.

mcm001 commented 7 months ago

But yeah the frontend seems to be super straightforward of a project as things go. I'll try sql first and if that's too slow, move to files in a folder?

mcm001 commented 7 months ago

Backend recipe proposal:

The current sql db can store a list of known ML model relative paths and their type/backend. On startup the NNModelManager can add any new ones that appear as UNKNOWN or something as well. Users would be able to see the model name in the UI and set the backend (eg RKNN) and type (eg, YOLOV5)