Can I detect objects from a live stream of video using a HDMI Capture Card?

[Flodkozza]

I see you can use a webcam for live detecting but is it possible to use a video capture card ( takes hdmi signal and plugs into the pc with USB ). It shows up in OBS studio as a source for example.

[AlexeyAB]

If your videostream can be captured by using default cv::VideoCapture OpenCV functions then just use Darknet out of box.

If you must have custom SDK, then you can build Darknet as SO/DLL lib and see example how ZED-camera SDK is used, just change it to your own SDK: https://github.com/AlexeyAB/darknet/blob/master/src/yolo_console_dll.cpp

[Flodkozza]

Thanks for the answer. Btw what is the recommended number of images for training of 2 classes that I need to be really really accurate?

Also can I make it so it crops the images from the web cam and not resize it? It seems like its better at detecting small objects when it crops it. ( crop around center )

Is it ok to train it with one set of images then when I get new images I just continue the training with them? Or its better to start over with all images at once?

[Flodkozza]

My FPS with webcam is 15-25, and it has lags that are like 500ms. OpenCV is built without CUDA, is that the problem?

[AlexeyAB]

You should build OpenCV with GStreamer.

Also build Darknet with LIBSO=1 and run this command LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH ./uselib data/coco.names cfg/yolov3.cfg yolov3.weights web_camera

[Flodkozza]

I dont use linux but that looks like a linux command.. I am using windows 10, is there a command for windows?

[AlexeyAB]

Compile yolo_cpp_dll.sln and yolo_console_dll.sln And run yolo_console_dll.exe data/coco.names cfg/yolov3.cfg yolov3.weights web_camera

[Flodkozza]

I tried building it with streamer and running that command and it still has a lot of lag, I dont think its fps problem, but it lags behind too much.

[AlexeyAB]

So this lag is generated by Web Camera (+ may be also OpenCV). Try to use another camera or Network camera with custom SDK.

[Flodkozza]

Its not the camera, I used a virtual web cam to test live detection and if I test it anywhere else its fine, so I guess its OpenCV. I can't fix that?

[AlexeyAB]

Also try to set here false instead of true and rebuild yolo_console_dll.sln https://github.com/AlexeyAB/darknet/blob/8c970498a296ed129ffef7d872ccc25d42d1afda/src/yolo_console_dll.cpp#L293

[Flodkozza]

Umm it says file not found when I run it but I didnt change anything except that boolean.

[Flodkozza]

I recompiled the yolo_cpp_dll and now it says Used GPU 0 and exits. Also couldnt compile for release because my opencv only had the debug .lib when I built it. Edit: Built the opencv for release and will try to compile everything for release now.

[Flodkozza]

When I run it now it says: C:\Users\hsky\Desktop\darknet\build\darknet\x64>yolo_console_dll.exe data/coco.names cfg/yolov3.cfg yolov3.weights web_camera Used GPU 0 Couldn't open file: v3.cfg Says v3.cfg even I entered yolov3.cfg. If I type in yolo.cfg it says: Couldn't open file: .cfg

[Flodkozza]

I fixed it by downloading the repo again and building it for Release. It's a lot faster now, thank you.

[Flodkozza]

I want to use darknet.exe and not the yolo console dll because I already edited some code and it doesn't work in yolo console dll for some reason. But if I do

bool detection_sync = true; // true - for video-file

in the darknet project, it doesn't change anything and it stills lags behind. Maybe a bug? In the yolo console, that setting changes it from like 50ms lag to 500ms lag but doesn't do anything in the darknet project.

[kmsravindra]

@AlexeyAB , Could you please explain what this variable "bool detection_sync = true; " is doing?

[AlexeyAB]

@kmsravindra

• if "bool detection_sync = true; " - then Detection will be done for each captured frame (better for video File)

• if "bool detection_sync = false; " - then Detection will be done for each FPS_Captured/FPS_Detection frame (better for real-time video Camera) - it allows to flush Video-Camera queue to reduce the latency

[Flodkozza]

Is 15 FPS too low? The delay of the capture is mostly gone now but the detection FPS is 15-18 and the detection boxes lag behind alot.

[AlexeyAB]

@Flodkozza It depends on your task.

• Either use tiny model instead of full model

• Or set width=320 height=320 in cfg file

• If you use yolo_console_dll.exe then you can set true for Kalman Filter: https://github.com/AlexeyAB/darknet/blob/8c970498a296ed129ffef7d872ccc25d42d1afda/src/yolo_console_dll.cpp#L291

• Or you can try to use Optical Flow tracker by un-commenting this line: https://github.com/AlexeyAB/darknet/blob/8c970498a296ed129ffef7d872ccc25d42d1afda/src/yolo_console_dll.cpp#L16

[Flodkozza]

Do some of these things reduce accuracy? I need it to be pretty accurate, I have a GTX 1060 3GB and it's going from 15-18FPS. Tiny model would decrease accuracy right?

[AlexeyAB]

width=320 height=320 and tiny model reduces accuracy.

Kalman filter and Optical Flow tracker don't reduce accuracy

[Flodkozza]

The Kalman filter creates a lot of duplicate detections if the camera is moving, and the comment does say it's for stationary cameras so I guess that doesn't help. I'm not sure for the tracking, it may improve tracking the objects when they are moving but if the camera moves they still lag behind alot. That's probably just the fps problem ? I noticed that my FPS goes from 15 to 25 if I have nothing running on my PC but it still looks too slow if the camera is moving. What FPS could I get on the tiny model with 320x320 on a 1060 3GB?

[NKM999]

rebuild Hello, I am new in this field. I am trying to run this on CPU offline.

How can I rebuild after un-commenting #define TRACK_OPTFLOW in darknet/src/yolo_console_dll.cpp file?

Kindly guide.