How to inference with model.half

[kbrajwani]

# torch.backends.cudnn.benchmark = True
# torch.backends.cudnn.enabled = True

from mmdet.apis import init_detector, inference_detector, show_result_pyplot
import mmcv
# Load model
config_file = 'cascade_mask_rcnn_hrnetv2p_w32_20e.py'
checkpoint_file = 'ICDAR.19.Track.B2.Modern.table.structure.recognition.v2.pth'
model = init_detector(config_file, checkpoint_file, device='cuda')
model.half()
img = "demo.png"

# Run Inference
result = inference_detector(model, img)

# Visualization results
show_result_pyplot(model, img, result, score_thr=0.85)

Hey i am using this code to reduce memory usage with fp16 but getting following error.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-4-d6c2e135ce10> in <module>()
----> 1 get_ipython().run_cell_magic('time', '', '# Test a single image \nimg = "demo.png"\n\n# Run Inference\nresult = inference_detector(model, img)\n\n# Visualization results\nshow_result_pyplot(model, img, result, score_thr=0.85)')

14 frames
<decorator-gen-53> in time(self, line, cell, local_ns)

<timed exec> in <module>()

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
    418                             _pair(0), self.dilation, self.groups)
    419         return F.conv2d(input, weight, self.bias, self.stride,
--> 420                         self.padding, self.dilation, self.groups)
    421 
    422     def forward(self, input: Tensor) -> Tensor:

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same

[hhaAndroid]

Hi，you can refer to test.py, it supports fp16 inference.

[kbrajwani]

Hey @hhaAndroid thanks it's works on fp16. I was converting model into fp16 to save memory but it's taking more memory that normal. can you please check?

At a time of model load.

After 2-3 image inference.

Can you tell me how can i resolve this issue.

[hhaAndroid]

@kbrajwani do you have any progress？