Open xhuvom opened 6 years ago
@xhuvom What version of OpenCV do you use?
I am using OpenCV version 2.4.13 and CUDA 8.0
Did you uncomment this line before compiling? https://github.com/AlexeyAB/darknet/blob/ea09a6e0b38e1ddf43ffcd81d27f0506411eb8e4/src/yolo_console_dll.cpp#L16
Tracking supports now only OpenCV 3.x, and this part doesn't support OpenCV 2.x So you should change this code for OpenCV 2.x if you want to use Tracking using optical flow.
I have tried to uncomment the line and recompile but no success. The error still persists. For a better clarification, shouldn't I comment out the line as I am using OpenCV 2.4.13 ? If I would use OpenCV 3.X then this line should work, am I right?
@xhuvom I fixed it. Just update code from this repo, leave //#define TRACK_OPTFLOW
commented, and recompile.
Yes, it should work with commented line on OpenCV 2.4.13. And should work with commented/uncommented line on OpenCV 3.x.
Thanks @AlexeyAB for the update. Meanwhile I have updated to OpenCV 3.4 and its working fine. However, the object tracking isn't robust yet. Any suggestion on tweaking the tracking feature for better performance?
You can tweak this params: https://github.com/AlexeyAB/darknet/blob/89d0c46bb32a98416fb55a9c83c291dd0831c6cc/src/yolo_v2_class.hpp#L166-L168
sync_PyrLKOpticalFlow_gpu->setWinSize(cv::Size(21, 21));
sync_PyrLKOpticalFlow_gpu->setMaxLevel(3);
sync_PyrLKOpticalFlow_gpu->setNumIters(2000);
But the increase in these parameters will be slightly slow down the optical flow.
I am using GTX1080 GPU showing Detection ~55 FPS and Capture ~70 FPS on videofile. To test the tracking for object count, I have increased the parameters like this:
sync_PyrLKOpticalFlow_gpu = cv::cuda::SparsePyrLKOpticalFlow::create(); sync_PyrLKOpticalFlow_gpu->setWinSize(cv::Size(21, 21)); // 15, 21, 31 sync_PyrLKOpticalFlow_gpu->setMaxLevel(10); // +- 3 pt sync_PyrLKOpticalFlow_gpu->setNumIters(10000); // def: 30
However, its still not reliable enough. Probably its the fundamental issue with the YOLO algorithm itself because if I dump the detected frames, it shows that not all objects are detected on each single frame.
Current implementation of tracking only used for track objects only until next detection. So if on one of detections the object isn't found then it will not tracked longer. Later maybe I will add more longer tracking.
It is required only if you Вetection speed much slower than Capture speed.
Thanks for your helpful suggestions @AlexeyAB !!. Maybe Kalman filter could be added for more robust tracking if the object is missing at some frames but reappears again.
Yes, Kalman filter can be added, Kalman filter takes into account the physical inertia of the movement of objects. But I use this tracking for my task, where vido-camera moves very chaotically, so Kalman filter can't be used.
May be I will take bounded boxes from Tracking and from Detection, then I will concatenate them and do NMS (Non-maximum-supression), instead of replacing Tracking-box by Detection-box. Also will try to filter tracking points by error and status: https://github.com/AlexeyAB/darknet/blob/89d0c46bb32a98416fb55a9c83c291dd0831c6cc/src/yolo_v2_class.hpp#L286
@AlexeyAB Thanks for your excellent repo and your continuing update. I encountered related errors while compiling "yolo_console_dll.sln". I’m using pre-build OpenCV3.4.0, CUDA8.0 and MSVS 2015. I set the project to Release x64, and configure the OpenCV include and lib path. The compiling is stopped with error: Can't open "opencv_cudaoptflow340.lib". Then I comment the "progma comment ..." lines below "opencv_world", no errors anymore and the .exe works fine. But I noticed that the output shows car ids, does this mean the tracking process is already working? I have not uncomment the line with “//#define TRACK_OPTFLOW” so far. After uncommenting the line with “//#define TRACK_OPTFLOW”, the compiling is stopped with error said that file “opencv2/cudaoptflow.hpp” can’t be found. I search the OpenCV folder and there is indeed no "cudaoptflow.hpp". Should I compile OpenCV with MSVS?
@yangulei Hi,
With commented //#define TRACK_OPTFLOW
- there isn't optical flow tracking. There is only tracking_id()
function, that shows track ID of objects.
With un-commented #define TRACK_OPTFLOW
- there is optical flow tracking, that helps to track objects in the realtime between detection. This is require if your GPU slow to detect objects for each frame in the realtime. To use #define TRACK_OPTFLOW
you should compile your OpenCV 3.x with CUDA support, for this:
C:/opencv_source/opencv
(so cmake file CMakeLists.txt
should be in the directory `C:/opencv_source/opencv/CMakeLists.txt
)C:/opencv_source/opencv
and where is binary C:/opencv_source/opencv/bin
Generate
Generate
C:\opencv_source\opencv\bin\OpenCV.sln
and build it with x64 & Release@AlexeyAB Thanks for your timing and clear replay. I'm compiling the OpenCV, which is really a time consuming process.
update: arise C2610 errors, building with BUILD_PERF_TESTS and BUILD_TESTS disabled according to https://github.com/opencv/opencv/issues/6716
@yangulei Thanks, I added BUILD_PERF_TESTS and BUILD_TESTS to my answer.
@AlexeyAB Cmake window, after pressing Generate
button, I can't seem to find opencv_cudaoptflow option to tick
Should I put some cuda files inside my opencv340/opencv/sources?
@bit-scientist Set checkboxes grouped
and advnaced
, then enter to the CUDA-group
Thank you, @AlexeyAB. Still couldn't find. Because there's no CUDA group to enter to. Maybe I should copy some cuda related files into C:/opencv340/opencv/sources?
I am trying to compile this fork on Ubuntu 16 with the following settings:
GPU=1 CUDNN=1 OPENCV=1 DEBUG=0 OPENMP=0 LIBSO=1
However, the compilation returns the following error:
In file included from /usr/local/include/opencv2/cudaoptflow.hpp:50:0, from src/yolo_v2_class.hpp:12, from src/yolo_v2_class.cpp:1: /usr/local/include/opencv2/core/cuda.hpp:158:33: error: ‘Stream’ has not been declared void upload(InputArray arr, Stream& stream); ^ compilation terminated due to -Wfatal-errors. Makefile:77: recipe for target 'darknet.so' failed make: *** [darknet.so] Error 1