Open ambitious-octopus opened 3 weeks ago
Hi @ambitious-octopus, once I'm able to reproduce the issue in #1186, I'll start to debug this issue.
Hi @ambitious-octopus , We have found the root cause for this error. We noticed that your model performs operations on constants, such as ”to” and “mul” operations, which cause failures in MCT. (specifically cause the model.to(device) error). To be more specific, I think those operations are done in the anchor preparation in your model.
This issue runs deeper, as manipulating constants during model inference can lead to accuracy degradation. Performing these manipulations in advance and using final constant values instead would enhance accuracy and reduce unnecessary calculations. Therefore, we recommend removing constant manipulations from the model and using the finalized constant values instead. This approach should also resolve issue 1189.
Idan
Issue Type
Bug
Source
pip (mct-nightly)
MCT Version
PR #1186
OS Platform and Distribution
Linux Ubuntu 22.04
Python version
3.10
Describe the issue
I'm attempting to quantize a YOLOv8n model from the Ultralytics package using MCT GPTQ. However, I encounter this error during the calibration process:
cc: @Idan-BenAmi
Expected behaviour
No response
Code to reproduce the issue
Dependencies:
Code:
Log output
No response