Problem about quantizing model with external module like DCNv2 and multi-head

[KiedaTamashi]

Hi, I am trying to apply torch2trt on FairMot model. It has an external library DCNv2.

1)With option fp16_mode=True, DCNv2 cannot be converted correctly and met a typeError.

RuntimeError: expected scalar type Float but found Half

By manually converting the data dtype, get a warning like following:

Warning: Encountered known unsupported method torch.Tensor.float Warning: Encountered known unsupported method torch.Tensor.float Warning: Encountered known unsupported method torch.Tensor.half

2)FairMot also has multiple heads and keeps outputs using dict dtype. But it seems not supported.

AttributeError: 'dict' object has no attribute '_trt'

namedtuple suggested by issue452 not works.

Codes: def init() self.NT = namedtuple('output', self.heads) ... def forward(self, x): x = self.base(x) x = self.dla_up(x) y = [] for i in range(self.last_level - self.first_level): y.append(x[i].clone()) self.ida_up(y, 0, len(y))

heads: hm/wh/id/reg

    z = {}
    if self.trt_flag:
        z = self.NT(*[self.__getattr__(head)(y[-1]) for head in self.heads])

Error Report: File "/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib/python3.7/site-packages/torch2trt-0.1.0-py3.7.egg/torch2trt/torch2trt.py", line 535, in torch2trt File "/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib/python3.7/site-packages/torch2trt-0.1.0-py3.7.egg/torch2trt/torch2trt.py", line 401, in mark_outputs AttributeError: 'output' object has no attribute '_trt'

Could you give any advice? Thank you.

[jaybdub]

Hi XiaoSanGit,

Thanks for reaching out!

Currently, the easiest way to get around the dict limitation, is to wrap or subclass the module. For example

class DictModule(nn.Module):

    def forward(self, x):
        return {'x': x}

class WrapperModule(DictModule):

    def forward(self, x):
        y = super(WrapperModule, self).forward(x)
        return y['x']

model = WrapperModule(...)
model_trt = torch2trt(model, ...)

Please let me know if this helps or you run into any issues.

Best, John

[KiedaTamashi]

@jaybdub Hi jaybdub, Thanks for your reply. I have bypassed this error by avoiding using dict before. But I then met

'[TensorRT] ERROR: INVALID_ARGUMENT: Cannot find binding of given name: input_0'

in the actual running.

But I haven't received a warning like 'Warning: Encountered known unsupported method xxxx'

So how can I know which part has problem? Is it the external module DCNv2?

Thanks for your time.

[minhhoangbui]

@XiaoSanGit Would you mind if I ask whether you succeed to make dcnv2 to work with tensorrt? At the moment, I fail to convert my CenterNet model to run in tensorrt :(