search

PhotonVision / photonvision

PhotonVision is the free, fast, and easy-to-use computer vision solution for the FIRST Robotics Competition.
https://photonvision.org
GNU General Public License v3.0
262 stars 166 forks source link

Support selecting Object Detection models #1359

Open Alextopher opened 5 days ago

Alextopher commented 5 days ago

It is going to take me a while to implement #1354 as I've got a lot to learn about the project and the tools :sweat_smile:

This (draft) PR is for part 1 of the implementation. It focuses on adding a model selection interface for models that exist in photonvision_config/models/. Upon completion we can ship more than 1 model and users could upload their own through ssh without deleting the shipped model.

TODO List

Removing documentation milestone from this PR because there are likely to be a lot of object detection changes this year making any docs I write out of date by the time the fall comes around.

mcm001 commented 5 days ago

Sadly I think I need to click the approve workflow button every time you push until you make at least one contribution

Alextopher commented 5 days ago

Sadly I think I need to click the approve workflow button every time you push until you make at least one contribution

Skill issue on my part. Sorry!

Alextopher commented 5 days ago

What do we think about the design for defining model labels?

note-640-640-yolov5s-labels.txt
note-640-640-yolov5s.rknn

My thought process is that someday when we are able to support uploading models in different formats we might see something like:

note-640-640-yolov5s-labels.txt
note-640-640-yolov5s.onnx
note-640-640-yolov5s.rknn
note-640-640-yolov5s.tflite
mcm001 commented 5 days ago

I think that potential conflict (with keeping track of label names/number of classes across different model types with the same name) makes me a little scared.

Alextopher commented 5 days ago

different model types

What I'm going for is all those files encode the same "model" - just in different file formats. The difference between onnx and rknn isn't topological, both have the same weights & structure, it's just they use a different format for encoding.

The metadata format I'm going for is name-width-height-model.format.

The alternative is you have multiple copies of the same model (keyed by '"name"`) and you risk screwing up writing the labels file.

Alextopher commented 5 days ago

I feel pretty good about the implementation so far. I have a question, how can I go about getting the list of available models from core to the client? I need to give the UI a List<String>.

Alextopher commented 3 days ago

Ready for review - if someone is able to test this code running on hardware that would be great.