azavea / raster-vision

An open source library and framework for deep learning on satellite and aerial imagery.
https://docs.rastervision.io
Other
2.03k stars 380 forks source link

Add ability to use different Objectdetection models than FasterRCNN #2147

Open julled opened 2 months ago

julled commented 2 months ago

🚀 Feature

It would be good to be able to use more up to date models than FasterRCNN in raster-vision. Torchvision offers some possible dropin replacements / alternatives like RetinaNet or SSD based detectors.

Motivation

More up to date models promise better performance at lower GPU needs.

Pitch

The object_detection_learner in raster-vision should be able to accept a configuration parameter which allows a different model to be used, as written above.

Alternatives

The user could do this manually in the code, but this is not very scaleable and unclear how many manual changes are needed.

Additional context

-

AdeelH commented 2 months ago

I agree that it would be useful to support other TorchVision object detection models. Adding something like an architecture field to ObjectDetectionModelConfig and then modifying ObjectDetectionModelConfig.build_default_model() might be the way to go.

That said, one way to do this without any changes to RV is to use the workaround described here: https://github.com/azavea/raster-vision/blob/3eb3ff1f8326a008460b9b37359950b552ea7bce/rastervision_pytorch_backend/rastervision/pytorch_backend/examples/object_detection/cowc_potsdam.py#L143-L180

You'll need to fork the torchvision repo and create a branch with hubconf.py modified like this: https://github.com/AdeelH/vision/blob/356a78a894acf2a377353a4cdb2df52c1cea84b4/hubconf.py#L68-L72