Open shanek16 opened 8 months ago
I don't think this behaviour is to do with quantisation. The trace of the error starts during your forward call to the quantised model...
# Calibrate the model
for data in calibration_loader:
with torch.no_grad():
output = prepared_model(data)
However, I am seeing the same error so there is still a problem here. I notice you are not applying the MiDaS transform in your code, This does not fix the issue for me but you will want to do this. Probably give the tutorial another read...
I have tracked down the issue (for me at least). I was making a copy of the model using the python built-in copy.deepcopy
method. The copy throws the error in the forward call, but the original works fine. I wouldn't be too surprised if the torch quantisation module does something similar, so this may well be the cause of @shanek16's issue.
Thank you for your work.
I am trying to quantize the MiDaS DPT_Large model into INT 8 quantization.
I have searched through github and googled, and asked bing if there is any one liner code to quantize the model into INT8 given calibration image folder.
However there seems no such way, and no examples or trials that other people had done to quantize MiDaS. I have tried quantization through torch.quantization module, but got the following error:
I used code:
I used Docker image: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel-23-04.html for environment setting.
I am having a hard time trying to quantize MiDaS manually by myself while I am still not familiar to pytorch functions. It would be a great help to know if anyone could share their story succeeding in quantizing MiDaS model. Any comments or advice is also welcome. Thanks.