Can't load an ONNX model in SSD Detector

[rjv-vila]

I want to change the model used in SSD Detector with my custom trained model which is an onnx and it is an facemask detector. Is it possible to use my model to detect facemask and the temperature of the person? I have modified the detector.py in SSD folder into this to load the my model : parent_dir_pth = os.path.dirname(os.path.abspath(file)) onnx_model_pth = ( parent_dir_pth + "/caffe/ssd-mobilenet.onnx" ) self._net = cv2.dnn.readNetFromONNX(onnx_model_pth)

but I get this error

Loading SSD weights from file... Traceback (most recent call last): File "main.py", line 220, in rgb_thread = RGBThread(model=FACE_DET_MODEL) File "/home/g1-facemask/ai-thermometer/rgb/rgb_thread.py", line 18, in init self._detector = SsdDetector() File "/home/g1-facemask/ai-thermometer/rgb/ssd/detector.py", line 18, in init self._net = cv2.dnn.readNetFromONNX(onnx_model_pth) cv2.error: OpenCV(4.4.0) /tmp/build_opencv/opencv/modules/dnn/src/dnn.cpp:604: error: (-2:Unspecified error) Can't create layer "433" of type "Exp" in function 'getLayerInstance'

I am using OpenCV4.4.0 and latest version of pytorch Is there a thing that I can do? Should I retrain my model that gets caffemodel and prototxt like the default? If yes, is there any tutorial to do that kind of training? @tomek-l

[tomasz-lewicki]

Hi @rjv-vila ,

Exp layer was added in OpenCV 4.5.2. by this PR: https://github.com/opencv/opencv/pull/19545

You will need to upgrade from OpenCV 4.4. All AI Thermometer functionality should still work under OpenCV 4.5.2.

[rjv-vila]

Thank you for your reply @tomek-l , should I upgrade it by building like in your instructions

wget https://raw.githubusercontent.com/mdegans/nano_build_opencv/master/build_opencv.sh chmod +x build_opencv.sh ./build_opencv.sh

or pip install will do?

[tomasz-lewicki]

You will need to build from source to have CUDA acceleration (so build_opencv.sh rather than pip install).

[rjv-vila]

Alright, thanks again. I will edit the version of the opencv written in the build_opencv.sh then build it. Is it the proper way of doing that? I saw that the default version of opencv written in build_opencv.sh is 4.4.0 so I though I just need to edit it.

I'll comment here an update if things goes fine or not. Thanks! :))

[rjv-vila]

@tomek-l here is what I get after upgrading to OpenCV 4.5.2

` Loading SSD weights from file... Weights loaded! Running first net inference... Detector initialized! GST_ARGUS: Creating output stream CONSUMER: Waiting until producer is connected... GST_ARGUS: Available Sensor modes : GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings: Camera index = 0 Camera mode = 0 Output Stream W = 3264 H = 2464 seconds to Run = 0 Frame Rate = 21.000000 GST_ARGUS: Setup Complete, Starting captures for 0 seconds GST_ARGUS: Starting repeat capture requests. CONSUMER: Producer has connected; continuing. [ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1081) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1 device opened! Version gpp: 3.3.26 dsp: 3.3.26 FLIR part #: b'500-0771-01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' FLIR serial #: b'\x02\xf1<\x01\x00\x00\x00\x00' format: b'UYVY' frame 160x120 @ 9fps format: b'Y16 ' frame 160x120 @ 9fps frame 160x122 @ 9fps format: b'Y8 ' frame 160x120 @ 9fps format: b'RGBP' frame 160x120 @ 9fps format: b'}\xeb6\xe4' frame 160x120 @ 9fps Estimated / selected altsetting bandwith : 18 / 642. INFO: Using $DISPLAY from environment, not from config (960, 0) exit handler called releasing CONSUMER: Done Success GST_ARGUS: Cleaning up GST_ARGUS: Done Success Traceback (most recent call last): File "main.py", line 244, in mainloop() File "main.py", line 142, in mainloop boxes_ir = transform_boxes(boxes, 1.1, 1.1, 0, 0) File "/home/g1-facemask/ai-thermometer/utils/transforms.py", line 49, in transform_boxes x1, y1, x2, y2 = bbox ValueError: not enough values to unpack (expected 4, got 1) `

Here is a copy of my onnx model if you want to try. Thanks! https://dropmb.com/EUixz Here is the label if needed. https://dropmb.com/DFwQP

[tomasz-lewicki]

Hi @rjv-vila !

Your Neural Net likely has a different output format than the network I used, so the existing code won't work. To make it work, you you'd have to parse the output of your neural network to extract the bounding boxes from the output tensor.

[rjv-vila]

Hello @tomek-l thank you again for replying.

Do you have any idea on how to do that? I'm kinda clueless on how I will do that process.