use Yolo3-Caffe with opencv4

[hamzeah]

Hello, I've already used Yolov3, original darknet model with python + OpenCV4 (Ref). but when I come to your caffe model, I get below error, exactly after performing net.forward(). error: (-2:Unspecified error) Can't create layer "Interp202" of type "Python" in function 'getLayerInstance'

Is there a way to solve this, without installing "caffe" and downgrade to OpenCV 3?

[jasonlovescoding]

Hi @hamzeah Have you uncommented `

WITH_PYTHON_LAYER := 1

` before compiling caffe?

[hamzeah]

Hello @jasonlovescoding I should thank you for great work and following this thread. Yes, I'd seen that before compiling Caffe. I successfully run your demo. let me explain better:

Yolov3-Caffe is very useful as many platforms (like novel AI accelerator sticks, intel NCS2 which requires Intel OpenVino, and others like OrangePi AI stick only, or mostly, support Caffe model. Moreover, many people have problems in using Yolo3-darknet on such platforms (conversions don't work properly....). I guess if you apply some changes, many people can use your model.

Q.1: in your code, you used Caffe methods, like net = caffe.Net(model_arch_path, model_weight_path, caffe.TEST), however, it OpenCV4, there exist methods to read net: net = cv2.dnn.readNet(model_arch_path, model_weight_path ), or more specifics, such as readNetfromcaffe,.... why you did not use them? actually this is important, because the Net model returned by OpenCV4 methods have some attributes usable for OpenVino and AI sticks. For example see this thread on Intel forum about such methos.

Q.2: if you manage to use OpenCV read net methods, "import caffe" may not be necessary (I used some caffe models without importing caffe).

Q.3: original Yolov3-darknet-coco dataset, has 608x608 max dimension, but yours have 418x418. Did you down-scale it (just want to know, anyway, 418x418 is faster and enough precise).

Thanks

[hamzeah]

@jasonlovescoding I investigate it a bit more, and all things are because of that "caffe python layer". I also opened a question about it in OpenCV forum, where admin wrote some comments on it. I am not expert with DNN design,etc., but if it is possible to have a quick fix, that would be great!

[jasonlovescoding]

Hi @hamzeah Thank you for the update. My guess would also have been that the python layer may not be supported in opencv dnn. After all custom layer introduces external code dependency.

So far there may not be an easy fix since original caffe does not have interp layer. You may be able to find it in some third-party forks (the model conversion tool I wrote is actually tailored for one, but beware of the off-by-one error I mentioned in README) and try similar things (i.e. write a layer, register the layer) to integrate interp layer into opencv dnn.

The model itself is fully convolutional, so the input size is is actually flexible - it should be able to work if you modify the input size in the prototxt file, but it should be multiples of 32. The current size is the default value of the pytorch model I converted (YOLOv3-pytorch, which is adapted from Ayoosh Kathuria's blog)

[jasonlovescoding]

Closing due to inactivity.