CarsCouting Problem

[17520299]

Excuse me, profressor. Cars Counting included measured velocity of vehicle function? I haven't installed yolo_lib yet.Instead, I have a trained-model car detection by Yolo v4. So i can use it for Cars Counting? Thanks you, sir!

[Nuzhny007]

Yes, but it need some manual work. You must set pairs of coordinates for binding video frame in pixels to map in geographical coordinates: https://github.com/Smorodov/Multitarget-tracker/blob/master/cars_counting/CarsCounting.cpp#L423

1. Save one frame from video and open it in image editor (gimp, ms paint etc);
2. Open Google Earth and find place with camera;
3. find 4 key points on frame and on map (points aren't on one straight line!);
4. Save coordinates in pixels and in degrees into vectors framePoints and geoPoints.

[17520299]

Hi, prof! In module CarsCouting, How can i configure the Road Line to count car on road? and Is Configuration-4-key-point different with Road Line?

[17520299]

Hi, prof! In Cars Counting module, just number of all car and truck, etc across the line. I wanna show every kind of vehicles, can you show me where to config it in your code, prof? Thanks you so much!!!

[Nuzhny007]

Hi!

In module CarsCouting, How can i configure the Road Line to count car on road?

Here AddLine method: https://github.com/Smorodov/Multitarget-tracker/blob/master/cars_counting/main.cpp#L49

and Is Configuration-4-key-point different with Road Line?

Yes, it's different

I wanna show every kind of vehicles, can you show me where to config it in your code, prof?

You can add any type to white list: https://github.com/Smorodov/Multitarget-tracker/blob/master/cars_counting/CarsCounting.cpp#L355 Or you can remove all types of objects from white list

[17520299]

Hi, prof! Can you show me how to config in step 2 and 3 with Yolov4 trained-model,please? Thanks you so much.

[17520299]

Hi, prof! If i have a YOLOv4 trained-vehicle model, instead of config belong to 3 step as you told me. I link my yolov4 weights, yolov4.cfg and yolov4.names (YOLO_LIB=OFF) into command line, be the same with your example. I can run modules CarsCounting, can't i? Thanks you prof!

[Nuzhny007]

Hi! Yes, you can. A 3 steps from another issue are for custom detector. If you want any existing detector with custom model then just set a model from command line and select inference framework.

[17520299]

Hi, prof! https://github.com/Smorodov/Multitarget-tracker/blob/a235980ff7059d4d1941f7f4760724a98b568cf1/cars_counting/CarsCounting.cpp#L196 Please explain for me about const TrackingObject& track!!! What does it mean and it included what inside? Thanks you prof!

[Nuzhny007]

Here: https://github.com/Smorodov/Multitarget-tracker/blob/9e7c558535f890d3eecec4b395d869cc3e5fdb79/src/Tracker/track.h#L188 It a current state of the tracking object. And a full info about object contains CTrack class: https://github.com/Smorodov/Multitarget-tracker/blob/9e7c558535f890d3eecec4b395d869cc3e5fdb79/src/Tracker/track.h#L251

[17520299]

Hi!!!! Excuse me, prof. I don't understand some line in your code. There are :

cv::Mat colorFrame; capture >> colorFrame; // and m_trace.size()

what are they? and what value do they return? Thanks you so much, prof!

[Nuzhny007]

cv::Mat colorFrame;
capture >> colorFrame;

This is capturing a new frame from file/camera.

m_trace.size() Length of the trajectory

[17520299]

Hi, prof!!! https://github.com/Smorodov/Multitarget-tracker/blob/8282f258abe40e9ca9edb214809024b437cf9d30/src/Tracker/Ctracker.cpp#L56 Excuse me. You can explain for me Update() function, can't you? And how is the flow of data(video) through the source code ? In my thinking, every frame of the video is splited and go through carscounting.cpp. Example: frame --> gray scale --> draw detect --> drawtrack and velocity --> checkIntersection. Thanks you so much!

[Nuzhny007]

Example: frame --> gray scale --> draw detect --> drawtrack and velocity --> checkIntersection

Gray scale don't used for neural networks based detector. In cars counting used a simple serial pipeline:

1. Capture frame
2. Detector -> regions with bounding box, type and score
3. Tracker.Update -> solving assignment problem with Hungarian algorithm or bipartite graph matching (new regions and old tracks), Kalman filter for predict a new coordinates
4. Draw results
5. Intersection

In another examples I used more complicated pipeline with multithreading.

[17520299]

Hi, prof! I have a same problem with CarsCounting modules in 2 computer(one have cuda(Yolo_lib=on) and the other has not(yolo_lib=off). I have yolov4's weight, cfg, names in hard drive of both of PC. 2 Computer gave me a same result(OpenCL not used). Can you explain this problem, prof! Thanks you very much!

[Nuzhny007]

Use -inf=opencv command line parameter

[17520299]

Hello!! It's the same return, prof. Do you think cfg, names file and libgtk2.0-dev be problem?

Use -inf=opencv command line parameter

[Nuzhny007]

1. Hm. Do you work through ssh? If not then lets install libgtk or qt, reconfigure opencv and rebuild it.

2. You can rebuild only this project with -DSILENT_WORK=ON

[17520299]

1. When i set -DSILENT_WORK=ON and ssh is installed. It can't build CarsCounting successfully. if -DSILENT_WORK=OFF, it is successful.
2. In PC which dont have cuda, i set YOLO_LIB=OFF although I have weights, cfg, names of YOLOv4 but it returns Darknet inference engine was not configured in Cmake. That mean I must set YOLO_LIB=ON to use CarsCounting? And i need to install darknet from alexeyAB in my PC?
3. VideoExample(face and pedestrian) has worked normally.
4. what package is nescessary to build CarsCounting successfully in case i have PC with nvidia's GPU(OpenCV, OpenCV_contrib, cuda, cudnn)?

Sorry if my content is messy. Thanks you very much, prof!

[Nuzhny007]

Now I'm not at home and have only phone with slow mobile internet. And can't check this case. But the error on your previous screenshot says that you need install any graphical library. For example liggtk2.0-dev, reconfigure OpenCV in CMake with gtk, rebuild it. And all will work

[17520299]

Hi, prof! I have run CarsCounting module successfully, but there are 2 problems which occur in output video.

1. There are no type of objects which belong to each bbox.
2. Function velocity measurement cant work.

Now, i use my custom yolov4 weights instead of pre-trained from alexey. Do you think these problem cause the bug in my case?? Thanks you prof!

[17520299]

Hi, prof! I have run CarsCounting module successfully, but there are 2 problems which occur in output video.

1. There are no type of objects which belong to each bbox.
2. Function velocity measurement cant work.

Now, i use my custom yolov4 weights instead of pre-trained from alexey. Do you think these problem cause the bug in my case?? Thanks you prof!

I know why that problem occurs because i didn't set geoCoordinate.(velocity measurement) Thank you so much.

[17520299]

Hi, prof! I has set geoCoordinate but it still cant display velocity measurement function. Would you mind telling me what happened, please! Thank you so much!

[Nuzhny007]

Hi! Enable code here - set #if 1 instead #if 0: https://github.com/Smorodov/Multitarget-tracker/blob/master/cars_counting/CarsCounting.cpp#L421

And here you can add advanced logs in stdout to control that all parameters is correct: https://github.com/Smorodov/Multitarget-tracker/blob/master/cars_counting/CarsCounting.cpp#L236

[17520299]

Hi!

And here you can add advanced logs in stdout to control that all parameters is correct: https://github.com/Smorodov/Multitarget-tracker/blob/master/cars_counting/CarsCounting.cpp#L236

What does it mean prof?? In this case, what are all parameters ??? I hasn't understood yet. And i have a question about Use GPU while run CarsCounting.

1. since i want to increase speed of it, i think that my laptop is running without GPU and cudnn_half.
2. when i use my custom yolov4 model, the speed of modules is decreased so much while yolov3-tiny in data folder is vice versa. Thank you so much prof!

[Nuzhny007]

You can print into stdout period, distance etc. If they are correct then we will search a problem in another area. Some parameters witch do you want to see on frame. I can't now write more logs - only on next week.

[Nuzhny007]

About performance: check your model with Multitarget-tracker example with tensorrt backend and set option -a=1 In this case you can get a maximum speed

[17520299]

Hi! I understand what you say. I can run your whole CarsCounting project successfully. Thank you for your support so much. Best regard!

[17520299]

About performance: check your model with Multitarget-tracker example with tensorrt backend and set option -a=1 In this case you can get a maximum speed

How about yolo_lib prof! I can do it the same with -a=1, can't i? let i try it!

[Nuzhny007]

Hi! I'm now with my laptop. And yes - you can try use Multitarget-tracker with any backend (darknet, tensorrt, opencv_dnn etc) and with -a=1 it will up to 2 times faster. If this result will prefer for you then I can refactor CarsCounting with some parallel pipeline

[17520299]

Hi, When i use pre-trained yolov3, speed of video is faster than my custom yolov4. although my yolov4 only have 4 classes and i have set -a=1. Do you think what happened? My hardware is the problem??? Ryzen 5 3550H and GTX 1650 max-Q Thank you prof!

[17520299]

Hi,

I wonder if a number of vehicles are too much as this, the fps is slow? There are any idea for increasing fps in this case, prof! Thank you so much!

[Nuzhny007]

It will be great to see:

1. GPU monitor: nvidia-smi -l
2. CPU monitor: plot or htop result
3. Tracker settings

And one more question: although my yolov4 only have 4 classes and i have set -a=1.

This option used for CarsCounting example or Multitarget-tracker? Now only can parallel processing detection and tracking

[17520299]

Hi,

I used this option for CarsCouting, prof!

[Nuzhny007]

I used this option for CarsCouting, prof!

Ok, I'll make it for CarsCounting also. And it will be faster

[Nuzhny007]

Hi! A big portion of refactoring CarsCounting module: https://github.com/Smorodov/Multitarget-tracker/pull/307 Read, plz, description of this PL and Files changed tab. Save your code before pulling a new updates.

[17520299]

Hi prof, I have a question to discuss with you. Firestly, In CarsCoutning, if an vehicle try to turn left, right or back, the trajectory will be wrong. Secondly, the performance of GPUs in my laptop is so low. it only use 10-15% when i check it in task manager. If i wanna increase performance of GPUs upto 40% or more to improve CarsCouting's result, it is possible? Can you show me some keywords or another tracking algorithm to find out the way to solve these problem? Thanks you so much!

[Nuzhny007]

Hi!

1. It need to enable advanced debug info: Pull the latest changes from this repository. For enabling advanced info in CTracker.cop set: #define DRAW_DBG_ASSIGNMENT 1 Here: https://github.com/Smorodov/Multitarget-tracker/blob/master/src/Tracker/Ctracker.cpp#L143 And after run with option -a=0, show me debug video or screen shots with wrong tracking.

2. Task manager don't show correctly GPU utilization. You need to use nvidia-smi utility, run it from command line: nvidia-smi -l So on my laptop with Windows Intel core i7 + Nvidia GeForce 1060M:

GPU utilization about 70% with command line: MultitargetTracker.exe D:\data\vehicles\Relaxing_highway_traffic.mp4 -ef=0 -w=C:\work\home\mtracker\Multitarget-tracker\data\yolov4.weights -c=C:\work\home\mtracker\Multitarget-tracker\data\yolov4.cfg -inf=tensorrt -e=7 -a=0 -g=0

GPU utilization about 80% with command line: MultitargetTracker.exe D:\data\vehicles\Relaxing_highway_traffic.mp4 -ef=0 -w=C:\work\home\mtracker\Multitarget-tracker\data\yolov4.weights -c=C:\work\home\mtracker\Multitarget-tracker\data\yolov4.cfg -inf=tensorrt -e=7 -a=1 -g=0

[Nuzhny007]

And I think if you will not show and update the result window then GPU utilization will be more.

[17520299]

Hi prof! Firstly, These are some wrong trackings, you can see that the red line is drawn in the sky, purple line is in the rightside and the green one is in line 0 direction. Secondly, when i rebuild newest version, i encountered a problem that it cannot detect vehicle like this. can you explain me what happened? Finally, the last time i checked GPU utilization is 84% by nvidia-smi. Thank you so much!

[Nuzhny007]

Hi! And whats about new debug window? Set, plz, #define DRAW_DBG_ASSIGNMENT 1 and show screen from this debug windows. And a part of log from console will be helpful. I need some advanced information for bugs fixing

[Nuzhny007]

Ok, it still not enough. More logs: Multitarget-tracker.exe <... your params...> 1>1.txt 2>2.txt And after closing program show, plz, a full content of the 1.txt and 2.txt And all options from command line.

[Nuzhny007]

Hi! I'm using default settings: https://github.com/Smorodov/Multitarget-tracker/blob/master/example/CarsCounting.cpp#L218 Or for concrete place with camera - concrete parameters

[17520299]

Hi! I see some elements in the dbgAssignment window. while the green, blue rectangle, and the red circle belong to each others is the tracking true. On the other hand, it is wrong. And the pink rectangle is the last state which the detector save in the window, isn't it? Why is detector not able to detect all vehicle in frame? My model is problem? or what? Please explain me, prof! Thanks you so much.

[Nuzhny007]

Hi! Red circle - current track size and position. Margenta rectangle - latest detection from model on previous frames (tracking work only on Kalman prediction). Green rectangle - current detection assigned to track. Blue rectangle - current detection assigned to track but distance between track and detection is too big (dist > m_distThres).

I think, it can be select good parameters for your case, May be:

m_trackerSettings.m_kalmanType = tracking::KalmanLinear;
    m_trackerSettings.m_filterGoal = tracking::FilterRect;
    m_trackerSettings.m_lostTrackType = useDeepSORT ? tracking::TrackNone : tracking::TrackSTAPLE; // Use visual objects tracker for collisions resolving. Used if m_filterGoal == tracking::FilterRect
    m_trackerSettings.m_matchType = tracking::MatchBipart;
    m_trackerSettings.m_useAcceleration = false;                   // Use constant acceleration motion model
    m_trackerSettings.m_dt = m_trackerSettings.m_useAcceleration ? 0.05f : 0.4f; // Delta time for Kalman filter
    m_trackerSettings.m_accelNoiseMag = 0.2f;                     // Accel noise magnitude for Kalman filter
    m_trackerSettings.m_distThres = 0.8f;                         // Distance threshold between region and object on two frames
    m_trackerSettings.m_minAreaRadiusPix = -1.f;
    m_trackerSettings.m_minAreaRadiusK = 0.8f;

But I don't sure - need make some experiments.

The next big improvement - make tracking not in pixels coordinates on frame but on geo coordinates on Earth plane. It need a big portion of changes and refactoring. I'm planing this work in the nearest time.

[17520299]

Hi prof! My yolov4 cannot detect all vehicles in frame althrough i fined-tune some parameters TrackerSettings. Do you think that my model is not good enough? I have trained 600 pictures and 100 for test. Thanks you so much!

[17520299]

In latest release, i see some detections which have tag "unknown". I dont understand about it!

[Nuzhny007]

Hi!

1. For my test I'm using default pretrained YOLOv4 from AlexeyAB repository. But if you has frame with size significantly greater than input layer then you can split frame to crops and make detection. Detector has parameter config.emplace("maxCropRatio", "-1"). Instead -1 you can set multiplier coefficient for crop size. For example: maxCropRatio = 2 If input layer has size 608x608 then crop size = 1216x1216. And FullHD frame 1920x1080 will be splatted to 4 crops with overlap. It will be slowed but more correctly.

2. "unknown" type may be if your types absent in this enum: https://github.com/Smorodov/Multitarget-tracker/blob/master/src/common/object_types.h#L6 Add your type after obj_vehicle. And here his name: https://github.com/Smorodov/Multitarget-tracker/blob/master/src/common/object_types.cpp#L85

[17520299]

Hi prof! i use -bs=2 in carscounting and it cannot detect any vehicles on frame! Can you show me why? Thanks you so much!!!

[Nuzhny007]

Hi! Thanx, it was a bug. Fixed: https://github.com/Smorodov/Multitarget-tracker/pull/325

[17520299]

Hi, prof! when i dive in Kalman Filter and Hungarian Algorithm, i have some questions.

1. can you explain me some parameters in your code about kalman filter as m_accelNoiseMag, m_minAreaRadiusPix, m_minAreaRadiusK? and m_distThres is IoU threshold for comparing similarity of object between 2 frame continuosly?
2. you only use IoU to compare similarity of object between 2 frames or another scores as shape score and convolutional cost?
3. in case, i have 1080p video, if i reduce its resolution to 720p, performance is possible to increase? ps: i set maxcrop=4 with 1080p, some vehicles cannot be detected.
4. there're so many times the ID switch occur if 2 or 3 objects overlap to each other, i think that only IoU is not good enough. Thanks you so much, and sorry if my english is bad!!!