ENet for semantic segmentation

[mmpinso]

Hello, would it be possible to have ENet inCaffe2? ENet seems perfect for real time image segmentation on smartphones due to its excellent compromise between results and latency. Caffe implementation by TimoSaemann. Tensorflow implementation by Kwotsin. Which cannot be deployed into CPU-only powered phones because of max_pooling_with_argmax kernel not being implemented for CPU by Tensorflow => maybe a potential differential added value for Caffe2 with respect to Tensorflow?

[Maratyszcza]

As far as I know, there are no plans to add ENet to Caffe2 model zoo, but if you are willing to contribute it, you are more than welcome! ENet is a pretty old model, and can't compete in quality with recent research, such as Mask R-CNN. Facebook AI Research recently released implementation of Mask R-CNN (and many other state-of-the-art models) for Caffe2. Last year at NIPS we demonstrated running this architecture in real-time on a mobile phone (see video here) through Snapdragon NPE backend of Caffe2.

cc @orionr

[orionr]

Hi @mmpinso. Mask R-CNN is your best segmentation solution here. It's still a bit of a challenge to get it running in real-time on device (we needed to use SNPE for example), but things are only getting faster. There are also some UNet architectures you could try and, if you only need detection, you can try YOLO 9000 for which there are some Caffe models you could convert to Caffe2. Good luck!

[orionr]

Also, for more details on Mask R-CNN2Go (a mobile-specific version of Mask R-CNN) check out https://research.fb.com/enabling-full-body-ar-with-mask-r-cnn2go/

[gustavz]

@orionr Is the Mask R-CNN2Go Code available somewhere?

[jax79sg]

Hi, i saw the Mask RCNN is integrated with Detectron, but the mobile specific version of it wasn't. Would Mask R-CNN2Go be open-sourced like it's predecessor, or if the caffe2 model be shared for NPE usage? Thanks.