OpenCV integration

[albertofernandezvillan]

Is it possible for a trained model with YOLONAS be used with OpenCV using, for example, the function cv::dnn::readNetFromONNX()

[dagshub[bot]]

Join the discussion on DagsHub!

[ofrimasad]

I am unfamiliar with this function, but the answer is that the ONNX produced by YOLO-NAS is just a basic ONNX, so it could be used by any framework supporting the layers used in YOLO-NAS. Please note that the quantized ONNX is specific for TensorRT, so you need to create a basic ONNX.

[BloodAxe]

YOLONAS supports native export to ONNX format:

from super_gradients.training import models
pretrained_model = models.get("yolo_nas_s", pretrained_weights="coco").cuda().eval()

dummy_input = torch.randn(1, 3, 640, 640).cuda()

yolo_model.prep_model_for_conversion(input_size=(640, 640))
torch.onnx.export(yolo_model, dummy_input, "yolo_nas_s.onnx", verbose=False, opset_version=13, do_constant_folding=True)

You can use generated ONNX file to load into TRT or ONNXRuntime as is.

Regarding OpenCV - I'm afraid you cannot use it "as is", due to some error inside ONNX parser on OpenCV side:

torch.onnx.export(yolo_model, dummy_input, "yolo_nas_s.onnx", verbose=False, opset_version=13, do_constant_folding=True)
>onnx_model = cv2.dnn.readNet("yolo_nas_s.onnx")
E           cv2.error: OpenCV(4.7.0) D:\a\opencv-python\opencv-python\opencv\modules\dnn\src\onnx\onnx_importer.cpp:1073: error: (-2:Unspecified error) in function 'cv::dnn::dnn4_v20221220::ONNXImporter::handleNode'
E           > Node [Constant@ai.onnx]:(onnx_node!/heads/Constant_11) parse error: OpenCV(4.7.0) D:\a\opencv-python\opencv-python\opencv\modules\dnn\src\onnx\onnx_graph_simplifier.cpp:848: error: (-215:Assertion failed) !field.empty() in function 'cv::dnn::dnn4_v20221220::getMatFromTensor'

In theory, running model through onnxsim or lowering opset version during export can help, but we haven't tested this and leave this to community to figure out since ONNX support in OpenCV is beyond our control.

I hope this answers your question.