search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.65k stars 7.96k forks source link

YOLOv4 integration with the Intel RealSense camera #7995

Open VAIBHAVPATEL97 opened 3 years ago

VAIBHAVPATEL97 commented 3 years ago

Hello, Is it possible to use Intel RealSense camera for real-time camera input to the YOLOv4 model?. I am trying to achieve this idea on Jetson TX2 or Jetson AGX Xavier. I did some research on google but was unable to find any leads. Any suggestions or comments is appreciated. Thanks in advance.

stephanecharette commented 3 years ago

Is there anything super special about that camera that would prevent it from being used?

VAIBHAVPATEL97 commented 3 years ago

This camera is capable of taking infrared, depth, and RGB pictures. I would like to use only RGB video to test the video but the YOLOv4 model chooses the Infrared monochrome video setting by default whenever I connect this camera. So, is there any python script that I can change within the repo so that I can get RGB video stream in real-time?

stephanecharette commented 3 years ago

Have you looked at the python script?

VAIBHAVPATEL97 commented 3 years ago

Yeah I was able to figure out and get working now I want to change the resolution of every test run which I do. Any suggestion for that @stephanecharette. Thanks for your quick response.

stephanecharette commented 3 years ago

Suggestions for what? For changing the camera resolution? I have no idea, I've never even googled that camera. Nor have you told us what you are using. I assume OpenCV? If so, it has well-documented API for changing the camera resolution.

VAIBHAVPATEL97 commented 3 years ago

I am trying to use this camera (Intel RealSense Camera) for real-time detection on the YOLOv4 model. So, to compare the performance of the trained model on different resolutions I want to change the resolution of the video captured by the camera. There is a darknet_video.py file present in the darknet repository which I am trying to change the achieve a different resolution. So, my question @stephanecharette to you is am I going correct or there is any other python file present in the darknet repo which can be changed to achieve the above mentioned task.

stephanecharette commented 3 years ago

I've never used any of the python files, so I really don't know. I'm more of a C++ developer. The usual way I would modify the camera resolution using OpenCV (with the eventual goal of feeding it into Darknet) would be with the cv::VideoCapture::set(cv::CAP_PROP_FRAME_WIDTH, ...) and CAP_PROP_FRAME_HEIGHT.

BUT all this is a moot point. Even if your camera could record frames at a resolution of 9999x9999, it doesn't matter. Darknet will resize every frame to match the network dimensions. So if your network is defined as 416x416, then you may as well set the camera to whatever resolution closely matches 416x416. Anything more than that is wasted, since the first thing Darknet will do is resize your large image to match the network dimensions.

VAIBHAVPATEL97 commented 3 years ago

I am looking into the OpenCV code present in the darknet_video.py for the camera resolution. What you explained above is completely true since I am also tuning the dimensions of the network to match the input video resolution to see whether or not it has any effect on the performance of the network (i.e. inference time). Thanks @stephanecharette for all your help.

Zhiying-Li-dot commented 2 years ago

@VAIBHAVPATEL97 Hello, I want to use Intel RealSense camera for real-time camera input to the YOLOv5 model. Could you please tell me how to get the stream? I set the camera as 0 when I connect my computer by USB3.0. But it is not working in Win10.

VAIBHAVPATEL97 commented 2 years ago

@lizhiying-2019 You can try to change the port number to 2 or 3 depending on the corresponding port number on your PC. Did you check the port number to which you camera is connected to the PC?