Add support for human and vehicle classes in megadetector

drivendataorg / zamba

A Python package for identifying 42 kinds of animals, training custom models, and estimating distance from camera trap videos

https://zamba.drivendata.org/docs/stable/

MIT License

106 stars 25 forks source link

Add support for human and vehicle classes in megadetector #106

Open pjbull opened 3 years ago

pjbull commented 3 years ago

We currently discard the clss information from megadetector since it used to just be animal/no-animal. As of version 4.1, there are person and vehicle classes: https://github.com/microsoft/CameraTraps/blob/master/megadetector.md#megadetector-v41-20200427

We should update megadetector to return the class information as well so that we can make use of these additional detection classes: https://github.com/drivendataorg/zamba/blob/69b397ab93ae868533ac9c98235a14146f133d27/zamba/models/mega_detector.py#L75-L78

r-b-g-b commented 3 years ago

Is this the right idea?

Currently student-teacher training only trains MDLite to detect the "animal" class.
With MegaDetector 4.1, we can now train it to detect "animal", "person", and "vehicle".
In our frame selection, we might now want to return frames with "animal" and "person" (since "human" is a zamba category). We would probably discard/not select frames with only "vehicle" labels, since that is not a zamba category. But still good to detect that class to help us not label videos as animal/person.
So we'd want to re-run the MegaDetector to label all of the videos, then re-train the MDLite student-teacher model with those categories.

pjbull commented 3 years ago

Yeah, this issue is just for returning the class information from the Megadetector here in the Zamba package since it's discarded. Let's move the MDLite training discussion to the algos repo so that we can point at the relevant code.