Open ashray21 opened 3 months ago
Hi, I think you have a few options:
Using polygraphy CLI: Example here illustrates that you could mention the calibration cache and rebuild the engine without needing to provide a calibration dataset this time. I'm referring to the section "[Optional] Rebuild the engine using the cache to skip calibration".
Command: polygraphy convert identity.onnx --int8 --calibration-cache identity_calib.cache -o identity.engine
Using Polygraphy API: Example here shows how you could define a custom calibrator with cache path mentioned. Please note that the documentation says this is the path you could not only save to but also load from.
Using the C++ API: For this, I think you'll need to supply a custom calibrator to the builder config with its readCalibrationCache() method defined so that the builder knows where to look. An old example illustrates this, but using the Python API: https://github.com/NVIDIA/TensorRT/issues/945
I have successfully generated calibrated dataset.cache file of my dataset using polygraphy. I want to load the generated calibrated cache file and create int8 engine using c++.
Function I'm using to convert onnx model into tensorrt engine file
@zerollzeng Please help