How to recognize the target of real-time video which return to the computer from drone?

[TimothyLiuu]

Hello! I'm a college student. Now I want to use the project to recognize the target of real-time video which return to the computer from drone. The drone which I used is Parrot bebop 2, it link the computer through Wifi. Now I can get the real-time video of drone on computer. But I don't know how to use the project to recognize the target of real-time video. I have tried ./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights -c 1 and ./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights http://192.168.42.1:44444 But the result is "Video-stream stopped!"

The IP Address and Port defined in my code:

define BEBOP_IP_ADDRESS "192.168.42.1"

define BEBOP_DISCOVERY_PORT 44444

Hope you can help me solve this problem, looking for u reply! @AlexeyAB

[AlexeyAB]

What video format / protocal does this dron use? What URL can you use to get videostream from Dron by using https://www.videolan.org/ VLC player?

[TimothyLiuu]

The official website shows that the drone use H264 video stream. I use 'Mplayer' to show the video stream on computer now, but I don't know it use which URL. Really thanks for replying me! @AlexeyAB

[AlexeyAB]

You should find URL.

[TimothyLiuu]

I checked a lot of information and now I can get video stream from drone by using VLC player. But it's using a file named 'a.sdp' instead of URL. Some answer says it can not get video stream by URL using VLC player.

The content of 'a.sdp' is: v=0 o=- 123456789 1 IN IP4 192.168.43.1 s=Bebop2 t=0 0 a=tool:sdp_test a=recvonly a=type:broadcast a=control:rtsp://192.168.43.1/video a=range:npt=0-0:12:34.567 a=rtcp-xr:pkt-loss-rle de-jitter-buffer m=video 55004 RTP/AVP 96 i=Front camera c=IN IP4 239.255.42.1/127 a=control:stream=0 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1;profile-level-id=420029;sprop-parameter-sets=Z2QAKKzZgHgGWwEQAAA+kAALuAjxgxmg,aOl488jw; m=video 55006 RTP/AVP 96 i=Vertical camera c=IN IP4 239.255.42.1/127 a=control:stream=1 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1;profile-level-id=420029;sprop-parameter-sets=Z2QAKKzZgHgGWwEQAAA+kAALuAjxgxmg,aOl488jw;

So how can I use the project to recognize the target of real-time video? @AlexeyAB

[AlexeyAB]

Try

./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights rtsp://192.168.43.1/video

or

./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights rtsp://192.168.43.1/video?dummy=param.mjpg

[TimothyLiuu]

I have tried these 2 commands, but it doesn't work. The difference is that it will stay at 'Video file: rtsp://192.168.43.1/video' for a while. But the result is still "Video-stream stopped!". Maybe I should try some other ways. Thanks for your help! @AlexeyAB

[AlexeyAB]

Try to do re streaming by using VLC as described there: https://github.com/AlexeyAB/darknet/issues/5170#issuecomment-611505911 open VLC and press CRTL+N, press File, add your 'a.sdp', press down-arrow near with Play, select stream, HTTP, select a port, and from darknet you connect to x.x.x.x:port, that is coming from VLC.

[TimothyLiuu]

Hi, I tried the method you say when I see your reply a week ago but I didn't success at that time. So I try to make it through other way these days. Today I try it again, firstly re streaming by using VLC through 'bebop.sdp' I write. The content of 'bebop.sdp' is: c=IN IP4 192.168.42.1 m=video 55004 RTP/AVP 96 a=rtpmap:96 H264/90000 And then use ./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights http://192.168.42.45:8080/video It worked! But the method has 2 problems: 1.The target recognition has servier delay about 2~3 minutes. 2.The computer suddenly shuts down when running the project. @AlexeyAB

[AlexeyAB]

1.The target recognition has servier delay about 2~3 minutes.

May be you should decrease buffer somewhere in VLC. 5-10 seconds is normal, but 2-3 minutes is too much.

2.The computer suddenly shuts down when running the project.

It seems some hardware issue, like overheating. Do you use Notebook or Desktop-PC?

[TimothyLiuu]

I use laptop to do the reserach, maybe it is a hardware problem caused by too much calculation. I wonder if there is some other way to realize the target recognition, because the method re streaming by using VLC is too complex and has servier delay. But I have no idea. There is a topic named 'How to use Yolo as DLL and SO libraries' in Readme of the project. I don't konw much about the SO libraries, it's that mean useing the Yolo project as a part of my project? Do you think this method will work? It's there some other way to use the project recognizing the target of real-time video without using command like: ./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights http://192.168.42.45:8080/video I feel you are really awesome and hot-hearted, if you have some idea, please tell me! @AlexeyAB

[AlexeyAB]

1. You can use Yolo as application: ./darknet detector demo ...

2. You can use Yolo as DLL(Windows) or SO(Linux) library from your C/C++ https://github.com/AlexeyAB/darknet/blob/master/src/yolo_console_dll.cpp or Python https://github.com/AlexeyAB/darknet/blob/master/darknet.py application

3. You can use Yolo just in OpenCV (on CPU, GPU, Myriad X, ...) https://github.com/opencv/opencv/blob/master/samples/dnn/object_detection.py or https://github.com/opencv/opencv/blob/master/samples/dnn/object_detection.cpp or in Vidia TensorRT-deepstream without Darknet at all for detection only

4. You can use Yolo in OpenDataCam https://github.com/opendatacam/opendatacam

5. You can use Yolo in TensorFlow https://github.com/mystic123/tensorflow-yolo-v3

[TimothyLiuu]

Hello, I have tried to use darknet_video.py to obtain the video stream, but it has some problem. cap = cv2.VideoCapture('./bebop.sdp') When I use my project to connect the drone and start the video-stream, I can use VLC through 'bebop.sdp' to play the video. But when I use darknet_video.py to obtain the video, it shows the error in the picture. @AlexeyAB