Per channel weight observer is not supported yet for ConvTranspose{n}d

[keremnymn]

I've trained a custom Mask RCNN model and I'm trying to export that to Torchscript. I have 'model_final.pth' file. This is the code I'm trying (I don't even know if this is true for custom training):

import copy
from detectron2.data import build_detection_test_loader
from d2go.export.api import convert_and_export_predictor
from d2go.utils.testing.data_loader_helper import create_fake_detection_data_loader
from d2go.export.d2_meta_arch import patch_d2_meta_arch

import logging

# disable all the warnings
previous_level = logging.root.manager.disable
logging.disable(logging.INFO)

patch_d2_meta_arch()

cfg_name = 'mask_rcnn_fbnetv3a_C4.yaml'
pytorch_model = model_zoo.get(cfg_name, trained=True)
pytorch_model.cpu()

with create_fake_detection_data_loader(224, 320, is_train=False) as data_loader:
    predictor_path = convert_and_export_predictor(
            model_zoo.get_config(cfg_name),
            copy.deepcopy(pytorch_model),
            "torchscript_int8@tracing",
            './',
            data_loader,
        )

# recover the logging level
logging.disable(previous_level)

The error I'm getting:

/usr/local/lib/python3.7/dist-packages/torch/quantization/qconfig.py in assert_valid_qconfig(qconfig, mod)
    126         )
    127         assert not is_per_channel, \
--> 128             'Per channel weight observer is not supported yet for ConvTranspose{n}d.'

AssertionError: Per channel weight observer is not supported yet for ConvTranspose{n}d.

[wat3rBro]

@keremnymn the default quantization backend doesn't support ConvTranspose, please set QUANTIZATION.BACKEND to "qnnpack".

[SuijkerbuijkP]

@wat3rBro after changing that parameter, running off the same code as above gives another error:

File "/home/pim/anaconda3/envs/d2/lib/python3.8/site-packages/d2go/export/api.py", line 100, in convert_and_export_predictor assert not fuse_utils.check_bn_exist(pytorch_model) AssertionError

Is there some bn setting that needs to be different for export as well?

edit:

Changing FBNET_V2 norm to "bn" and the others to "BN" (from https://github.com/facebookresearch/d2go/issues/39), does not help either.

[wat3rBro]

@SuijkerbuijkP I see, I think it's because the roi mask head is not built with FBNet builder and currently quantization is incompatible with that head, we're working on a fix.

[petoor]

I can reproduce @SuijkerbuijkP findings. Changing cfg.MODEL.MASK_ON = False in order to not use the roi mask head, the script seems to hang indefinitely . Both with and without cfg.QUANTIZATION.BACKEND = "qnnpack

[mahesh-sudhakar]

Hi @wat3rBro, Just writing here as I'm struck with the same Per channel weight observer is not supported yet for ConvTranspose{n}d error too.

I fine-tuned a mask_rcnn_fbnetv3a_C4.yaml model on my custom COCO style segmentation data set, and face the error while trying to export it as an Int8 model.

[wat3rBro]

@smahesh2694 Hi we've updated the mask_rcnn_fbnetv3a_C4.yaml (https://github.com/facebookresearch/d2go/commit/477ab964e2165cb586b5c00425f6e463d7edeadd) and now it should work with quantization using qnnpack. There's also a test for it https://github.com/facebookresearch/d2go/blob/2366ab940d6d87cc2b03f8a6c97d5fc9aed56c62/tests/modeling/test_meta_arch_rcnn.py#L39-L58

[petoor]

Hi @wat3rBro It is hard to test since patch_d2_meta_arch cant be imported in the current HEAD of master

[ys31jp]

I have the same issues when export with mask_rcnn_fbnetv3g_fpn.yaml. How can I solve this?