Hoping to get some advice on how to merge distiller's quantization part with mmdetection

[brisker]

Here I want to get some advice on how to merge distiller's quantization part with open-sourced projects mmdetection.( maybe not the entire mmdetection projects, but at least a demo that works) 1) First of all, is it better to add the distiller code to the mmdetection, or add the mmdetection code to distiller? (The main traininig file of mmdetection is like this: https://github.com/open-mmlab/mmdetection/blob/master/tools/train.py) 2) Currently, what I only know is that we should modify all the operations like eltwiseadd, concat, etc. in mmdetections into self_defined ops. Besides this, any other points to pay attention to? Can anybody give some basic advice? Thanks a lot in advance!

[levzlotnik]

Hi @brisker , sorry for the delay...

I'll answer your questions one by one -

1. We usually add the distiller code to the example code, it usually takes a couple of lines and you're good to go. Some good examples for this are compress_classifier.py and word_language_model/main.py, where we took the original code from PyTorch's sample applications:

   • imagenet classification training sample -> compress_classifier.py (at least before the last update to it)
   • word language model training sample -> word_language_model/main.py.

2. I am currently working on the object detection sample, there is another op that we need to implement as a layer if we want to quantize it, and it is RoIAlign (This operation takes an image and a list of RoIs with their respective coordinates and outputs a list of fixed size "sub-images" that come from these RoIs). That might be a bit problematic, because as you know - in mmdetection they actually took the code for RoIAlign from torchvision sources (which is written as an extension in C++ & CUDA), making it inaccessible to us if we want to quantize it like it would be done in actual hardware.

   Other than that - you need to make sure all the ops you want to quantize (especially in the case of Post Train Quantization) are defined as layers in your model, i.e. they are accessible by model.named_modules().

Let me know if you have more questions.

Thanks, Lev

[guyjacob]

@brisker, if you're still interested, you might want to look at the NCF example we just merged into master. It's simpler than the image classification sample but still has all the Distiller API calls required for post-training quantization and quantization-aware training. (Note that the word language model script actually does only pruning, so it's not the best example in your case).

Regarding question 2, beyond what @levzlotnik said, please check out this guide in our docs, it has more details in terms of changes required to the model.

[levzlotnik]

Hi, Closing this issue due to staleness. If more question arise - feel free to reopen.