SuperGlue and NetVLAD stuck with tensorRT

[allegorywrite]

Hi, I tried to speed up CNN inference using tensorRT, but the following two problems occurred. Have these issues been addressed?

SuperGlue stuck with warning:

[W:onnxruntime:SuperGlueOnnx, tensorrt_execution_provider.h:60 log] [2024-07-29 02:30:30 WARNING] - Subnormal FP16 values ​​detected.
[W:onnxruntime:SuperGlueOnnx, tensorrt_execution_provider.h:60 log] [2024-07-29 02:30:30 WARNING] If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
[W:onnxruntime:SuperGlueOnnx, tensorrt_execution_provider.h:60 log] [2024-07-29 02:30:30 WARNING] Weights [name=pred/vlad/memberships/Conv2D.weight] had the following issues when converted to FP16:

NetVLAD issues a warning and gets stuck when quantizing to int8:

[W:onnxruntime:, tensorrt_execution_provider.h:60 log] [2024-07-29 02:36:04 WARNING] Calibrator is not being used. Users must provide dynamic range for all tensors that are not Int32 or Bool. 
[W:onnxruntime:, tensorrt_execution_provider.h:60 log] [2024-07-29 02:36:04 WARNING] Missing scale and zero-point for tensor (Unnamed Layer* 1) [Shuffle]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor

[allegorywrite]

This problem has been solved. The folder for the provided CNN models contains engine caches such as TensorrtExecutionProvider_TRTKernel_graph_tf2onnx_406512749031481965_1_0_fp16_int8, but these cache files contain data that is strictly dependent on the hardware used, so loading them caused an error. To address this problem, you need to delete all engine files under models and rewrite the options in onnx_generic.h as follows to generate new cache files: tensorrt_options.trt_engine_cache_enable = 1;