search

hybridgroup / gocv

Go package for computer vision using OpenCV 4 and beyond. Includes support for DNN, CUDA, OpenCV Contrib, and OpenVINO.
https://gocv.io
Other
6.75k stars 869 forks source link

cuda and openvino with NVIDIA GPU ONLY WITHOUT INTEL GPU #814

Open alaayousef87 opened 3 years ago

alaayousef87 commented 3 years ago

Hi First really big thanks for you I learn a lot from your profile :) I have Ubuntu server with gpu cuda cudnn enabled and successfully configured and tested I want to use openvino but also with cuda my server have only NVIDIA gpu i dont't have any intel gpu and it will be difficult to have intel gpu because that's a dedicated server

my question in my case can I still used openvino with NVIDIA cuda without intel GPU in opencv? I hope if anyone can answer my quastion because I will be crazy I am trying doing that from one month but I am failed :(((

when i install gocv with cuda it is success and i have successfully gocv with opencv cuda enabled but when I install all to have openvino together with cuda I have the cuda success and also openvino but when I activate the opencv of openvino verstion by go cv openvino env environment file I lost the cuda :( and even the openvino the ie import failed last point I see in make file dldt but that's change to openvino so I am confused about this point also Thank you

Description

Steps to Reproduce

1. 2. 3. 4.

Your Environment

WendelHime commented 3 years ago

Hi @alaayousef87, could you add the steps to reproduce your issue? Which commands do you use to install gocv with openvino and cuda? make install_openvino and make install_cuda?

alaayousef87 commented 3 years ago

Hi Thank you for your response,

I use make install_all

Should I do it like that make install_openvino and install_cuda it is not the same when I call make install_all ?

Another question before I use make install openvino should I install the openvino toolkit by installer ? Or the script will also install it for me???

I don’t have any intel gpu so the cldnn option can I keep it enabled in make install openvino? If not what i should use in instead the cpu plugging??

In the makefile in the command to install all for opencv the inference engine path is going to dldt folder ? But how because in the install you do it in openvino folder Can you explain me this point please 🙏

Last question I did another approach:

Build openvino from source with disable gpu plugging and enabled cpu plugging

Build opencv from source with cuda enabled and inferences engine ngraph enable

And all of tha run successfully until here

When I run gocv after source to my custom environment the cuda working but the openvino not seeing the ie version , when I try to apply the command in the cmd openvino ie version give me error intferance engine not found

Thank you

On Wed, 24 Mar 2021 at 12:01 AM Wendel Hime @.***> wrote:

Hi @alaayousef87 https://github.com/alaayousef87, could you add the steps to reproduce your issue? Which commands do you use to install openvino and cuda? make install_openvino and make install_cuda?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hybridgroup/gocv/issues/814#issuecomment-805197877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE73ZBOIXP2JR6QR2VRJDGLTFDXPZANCNFSM4ZQSYE4A .

alaayousef87 commented 3 years ago

Hi again:)

I have one update I successfully have now opencv version working perfectly with cuda and with openvino :)) but not in gocv :((

Now in python when I try to run dnn with backend cuda is running perfectly with any models but not openvino model

Also when I run dnn with backend interface engine and cpu target with openvino model also work perfectly

But when I run dnn in openvino model and backend cuda it is not working and I believe that’s something I can’t do without gocv

Now my question is really gocv can do that for me i mean I can run openvino model in cuda backend with gocv ??

Really I appreciate any help answer or comment

Thank you

On Wed, 24 Mar 2021 at 12:01 AM Wendel Hime @.***> wrote:

Hi @alaayousef87 https://github.com/alaayousef87, could you add the steps to reproduce your issue? Which commands do you use to install openvino and cuda? make install_openvino and make install_cuda?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hybridgroup/gocv/issues/814#issuecomment-805197877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE73ZBOIXP2JR6QR2VRJDGLTFDXPZANCNFSM4ZQSYE4A .

WendelHime commented 3 years ago

Hi @alaayousef87 , could you provide some sample code with a openvino model? This way I can try to reproduce and check what is happening.

Now my question is really gocv can do that for me i mean I can run openvino model in cuda backend with gocv ??

I don't have this answer yet, but I have guess, if it isn't working with python lib, it can be something related directly to OpenCV C++ lib.

Please execute this method to check all available backends and print on response

import cv2
print(cv2.dnn.getAvailableBackends())
alaayousef87 commented 3 years ago

Hi,

import cv2 cv2.cuda.getCudaEnabledDeviceCount() 1 print(cv2.dnn.getAvailableBackends()) Traceback (most recent call last): File "", line 1, in AttributeError: module 'cv2.dnn' has no attribute 'getAvailableBackends'

getAvailableBackends is not in my dnn !! maybe this function for c++ lib not python? My sample code:

ARCH_FPATH_XML="intel/face-detection-retail-0004/FP16/face-detection-retail-0004.xml" MODEL_FPATH_BIN ="intel/face-detection-retail-0004/FP16/face-detection-retail-0004.bin"

Load the model.

net = cv2.dnn.readNet(ARCH_FPATH_XML, MODEL_FPATH_BIN)

print("Loading the model successfully by dnn lets set backend ..")

Specify target device.

DNN_BACKEND_INFERENCE_ENGINE

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)

net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)

NVIDIA GPU or Intel GPU as a target backen

DNN_TARGET_CUDA_FP16 DNN_TARGET_CUDA DNN_BACKEND_CUDA

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

print("Loading the model with backend what is my layers by dnn ..") ln = net.getLayerNames() print(ln)

ln = [ln[i[0] - 1] for i in net.getUnconnectedOutLayers()]

print(ln)

Read an image.

print("Processing input image...") IMG_FPATH ="firstimg.jpg" frame = cv2.imread(IMG_FPATH) if frame is None: raise Exception(f'Image not found here: {IMG_FPATH}') print("Loading the image with backend set successfully by dnn ..")

Prepare input blob and perform inference

blob = cv2.dnn.blobFromImage(frame, size=(672, 384), ddepth=cv2.CV_8U) net.setInput(blob) out = net.forward()

returned error Layer detection_out of type unsupported on OpenCV backend in function forwardLayer my target to solve that error to run openvino model in my cuda backend

if I changed to this 2 line net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU) it is working correctly with openvino model

the 2 line of setting backend cuda working fine with any other model "not openvino model" example of 100% working code with cuda using yolo:

USAGE

python yolo_object_detection.py --input ../example_videos/janie.mp4

--output ../output_videos/yolo_janie.avi --yolo yolo-coco --display 0

python yolo_object_detection.py --input ../example_videos/janie.mp4

--output ../output_videos/yolo_janie.avi --yolo yolo-coco --display 0 --use-gpu 1

import the necessary packages

from imutils.video import FPS import numpy as np import argparse import cv2 import os

construct the argument parse and parse the arguments

ap = argparse.ArgumentParser() ap.add_argument("-i", "--input", type=str, default="", help="path to (optional) input video file") ap.add_argument("-o", "--output", type=str, default="", help="path to (optional) output video file") ap.add_argument("-d", "--display", type=int, default=1, help="whether or not output frame should be displayed") ap.add_argument("-y", "--yolo", required=True, help="base path to YOLO directory") ap.add_argument("-c", "--confidence", type=float, default=0.5, help="minimum probability to filter weak detections") ap.add_argument("-t", "--threshold", type=float, default=0.3, help="threshold when applyong non-maxima suppression") ap.add_argument("-u", "--use-gpu", type=bool, default=0, help="boolean indicating if CUDA GPU should be used") args = vars(ap.parse_args())

load the COCO class labels our YOLO model was trained on

labelsPath = os.path.sep.join([args["yolo"], "coco.names"]) LABELS = open(labelsPath).read().strip().split("\n")

initialize a list of colors to represent each possible class label

np.random.seed(42) COLORS = np.random.randint(0, 255, size=(len(LABELS), 3), dtype="uint8")

derive the paths to the YOLO weights and model configuration

weightsPath = os.path.sep.join([args["yolo"], "yolov3.weights"]) configPath = os.path.sep.join([args["yolo"], "yolov3.cfg"])

load our YOLO object detector trained on COCO dataset (80 classes)

print("[INFO] loading YOLO from disk...") net = cv2.dnn.readNetFromDarknet(configPath, weightsPath)

check if we are going to use GPU

if args["use_gpu"]:

set CUDA as the preferable backend and target

print("[INFO] setting preferable backend and target to CUDA...") net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

determine only the output layer names that we need from YOLO

ln = net.getLayerNames() print("[lolo]what is my layers...") print(ln) ln = [ln[i[0] - 1] for i in net.getUnconnectedOutLayers()] print("[lolo]what is my layers. after..") print(ln)

initialize the width and height of the frames in the video file

W = None H = None

initialize the video stream and pointer to output video file, then

start the FPS timer

print("[INFO] accessing video stream...") vs = cv2.VideoCapture(args["input"] if args["input"] else 0) writer = None fps = FPS().start()

loop over frames from the video file stream

while True:

read the next frame from the file

(grabbed, frame) = vs.read()

if the frame was not grabbed, then we have reached the end

of the stream

if not grabbed: break

if the frame dimensions are empty, grab them

if W is None or H is None: (H, W) = frame.shape[:2]

construct a blob from the input frame and then perform a forward

pass of the YOLO object detector, giving us our bounding boxes

and associated probabilities

blob = cv2.dnn.blobFromImage(frame, 1 / 255.0, (416, 416), swapRB=True, crop=False) net.setInput(blob) layerOutputs = net.forward(ln)

initialize our lists of detected bounding boxes, confidences,

and class IDs, respectively

boxes = [] confidences = [] classIDs = []

loop over each of the layer outputs

for output in layerOutputs:

loop over each of the detections

for detection in output:

extract the class ID and confidence (i.e., probability)

of the current object detection

scores = detection[5:] classID = np.argmax(scores) confidence = scores[classID]

filter out weak predictions by ensuring the detected

probability is greater than the minimum probability

if confidence > args["confidence"]:

scale the bounding box coordinates back relative to

the size of the image, keeping in mind that YOLO

actually returns the center (x, y)-coordinates of

the bounding box followed by the boxes' width and

height

box = detection[0:4] * np.array([W, H, W, H]) (centerX, centerY, width, height) = box.astype("int")

use the center (x, y)-coordinates to derive the top

and and left corner of the bounding box

x = int(centerX - (width / 2)) y = int(centerY - (height / 2))

update our list of bounding box coordinates,

confidences, and class IDs

boxes.append([x, y, int(width), int(height)]) confidences.append(float(confidence)) classIDs.append(classID)

apply non-maxima suppression to suppress weak, overlapping

bounding boxes

idxs = cv2.dnn.NMSBoxes(boxes, confidences, args["confidence"], args["threshold"])

ensure at least one detection exists

if len(idxs) > 0:

loop over the indexes we are keeping

for i in idxs.flatten():

extract the bounding box coordinates

(x, y) = (boxes[i][0], boxes[i][1]) (w, h) = (boxes[i][2], boxes[i][3])

draw a bounding box rectangle and label on the frame

color = [int(c) for c in COLORS[classIDs[i]]] cv2.rectangle(frame, (x, y), (x + w, y + h), color, 2) text = "{}: {:.4f}".format(LABELS[classIDs[i]], confidences[i]) cv2.putText(frame, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)

check to see if the output frame should be displayed to our

screen

if args["display"] > 0:

show the output frame

cv2.imshow("Frame", frame) key = cv2.waitKey(1) & 0xFF

if the q key was pressed, break from the loop

if key == ord("q"): break

if an output video file path has been supplied and the video

writer has not been initialized, do so now

if args["output"] != "" and writer is None:

initialize our video writer

fourcc = cv2.VideoWriter_fourcc(*"MJPG") writer = cv2.VideoWriter(args["output"], fourcc, 30, (frame.shape[1], frame.shape[0]), True)

if the video writer is not None, write the frame to the output

video file

if writer is not None: writer.write(frame)

update the FPS counter

fps.update()

stop the timer and display FPS information

fps.stop() print("[INFO] elasped time: {:.2f}".format(fps.elapsed())) print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))

thank you

On Wed, Mar 24, 2021 at 6:53 PM Wendel Hime @.***> wrote:

Hi @alaayousef87 https://github.com/alaayousef87 , could you provide some sample code with a openvino model? This way I can try to reproduce and check what is happening.

Now my question is really gocv can do that for me i mean I can run openvino model in cuda backend with gocv ??

I don't have this answer yet, but I have guess, if it isn't working with python lib, it can be something related directly to OpenCV C++ lib.

Please execute this method to check all available backends and print on response

import cv2print(cv2.dnn.getAvailableBackends())

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hybridgroup/gocv/issues/814#issuecomment-805886858, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE73ZBJL535ZBKO2IVKPWWTTFH4GLANCNFSM4ZQSYE4A .

WendelHime commented 3 years ago

getAvailableBackends is not in my dnn !! maybe this function for c++ lib not python?

Yeah, I'm really sorry, I was looking at the docs and wasn't expecting this.

Following your error:

Layer detection_out of type unsupported on OpenCV backend in function forwardLayer

It seems that CUDA Backend don't have support to this layer, but looking at OpenCV implementation, this message only raises when preferableBackend is equal to DNN_BACKEND_OPENCV which is weird, even if you set your preferable backend it will use DNN_BACKEND_OPENCV (as defined here). I believe you'll have a better support for this case opening a issue on opencv/opencv

alaayousef87 commented 3 years ago

Hi

Thank you so much for your support

I have this error because of this condition it->second.empty because I see it is empty when I trace my code

Anyway the IR model from openvino will ever never run in cuda they are intel and they will run only on intel GPU and cannot run in nvidia GPU , I have this answer after difficult work and I like to share maybe other people will avoid to do that mistake and trying to do something not doable :)

Yes you can have one opencv version with cuda enabled and inference engine for openvino and you can run both of openvino and cuda in same opencv version but when you use the dnn to read and then forward to network You need to use the supported back end to each model

By example yolo can support backend cuda But IR Model from openvino can support cldnn the plugin of intel gpu but will ever never support the nvidia cuda gpu

Even if you try to convert the model to model can support cuda like tesnsorflow by example you can’t, you can convert tensorflow to IR model but you can’t transfer the IR model to tensorflow

This is my conclusion I hope if there is anyone having another information to share it with us

Thank you 🙏

On Wed, 24 Mar 2021 at 9:59 PM Wendel Hime @.***> wrote:

getAvailableBackends is not in my dnn !! maybe this function for c++ lib not python?

Yeah, I'm really sorry, I was looking at the docs and wasn't expecting this.

Following your error:

Layer detection_out of type unsupported on OpenCV backend in function forwardLayer

It seems that CUDA Backend don't have support to this layer, but looking at OpenCV implementation https://github.com/opencv/opencv/blob/master/modules/dnn/src/dnn.cpp#L3183, this message only raises when preferableBackend is equal to DNN_BACKEND_OPENCV which is weird, even if you set your preferable backend it will use DNN_BACKEND_OPENCV (as defined here https://github.com/opencv/opencv/blob/master/modules/dnn/src/dnn.cpp#L1181). I believe you'll have a better support for this case opening a issue on opencv/opencv https://github.com/opencv/opencv

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hybridgroup/gocv/issues/814#issuecomment-806039635, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE73ZBJA5FXWUJ6QQVDX7SLTFISBDANCNFSM4ZQSYE4A .