search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.33k stars 9.42k forks source link

Adjusting the output format of a model (mask2former) #8199

Open avickars opened 2 years ago

avickars commented 2 years ago

I am currently about 90% of the way there in converting mask2former into torchscript. The issue I am running into is how to adjust the output of the model as it outputs a list of numpy arrays that are of course incompatible with torchscript. I so far have been unable to find the code relating to the model output in the mm detection code base. My assumption is that since it is a pytorch model, it must be outputting a tensor and the conversion into numpy arrays is happening after the fact during a post-processing step. If someone could point me to the spot where that it is happening it would be greatly appreciated!

jbwang1997 commented 2 years ago

The forward and post-process code is both in the simple_test of maskformer detector. https://github.com/open-mmlab/mmdetection/blob/master/mmdet/models/detectors/maskformer.py#L111. You can use the outputs of mask2former_head forward which is without postprocess.

liuyunfeng2016 commented 2 years ago

@avickars @jbwang1997 Hello, have you successfully trained mask2former? Have you encountered any of the following errors: ValueError: matrix contains invalid numeric entries? For details, see #8155

avickars commented 2 years ago

yup I had no problems training, haven't seen that error.