DeepSORT with custom YOLOv5 model

[Guemann-ui]

Hi, could I use the DeepSORT model with a custom YOLOv5 model? Because I faced some error when I tried it. Thanks

[danielcrane]

You should be able to use Deep SORT with any detector that you want, as long as you ensure that the bounding boxes outputted from the detector are in the same format Deep SORT expects.

It might be helpful if you could share the error and some examples of detector outputs so people can give more useful guidance.

[cydni]

@danielcrane I'm currently working the the process of running the DeepSORT tracker and working on generating the detections using YOLOv5 are the detections typically in the correct format or is there some extra work that needs to be done to transform them? Thank you.

[danielcrane]

@cydni I'm not personally using https://github.com/nwojke/deep_sort/blob/master/deep_sort_app.py when I use Deep SORT (I've totally rewritten it for my own use-cases to take YOLO inputs on a frame-by-frame basis), but if you did want to use this repo as-is then according to the following lines you should save the detections in the MOT Challenge format along with feature vector information:

https://github.com/nwojke/deep_sort/blob/280b8bdb255f223813ff4a8679f3e1321b08cdfc/deep_sort_app.py#L96-L104

Note that this format is different from the YOLOv5 default format, but it's not too hard to rewrite it to take those inputs instead I don't think.

Also, if you only have the bounding box detections, and don't have a model that can produce meaningful feature vectors for those detections, then I'd advise checking out the regular SORT instead of Deep SORT, since the use of these feature vectors is the main difference between the two.

[cydni]

Ok thanks a lot! I don't have a such a model so I'll look into using Sort instead. Would you happen to know if that default format for YOLOv4 is closer to the necessary format?

[danielcrane]

Ok thanks a lot! I don't have a such a model so I'll look into using Sort instead. Would you happen to know if that default format for YOLOv4 is closer to the necessary format?

All YOLO models have the same output format, but it shouldn't be hard to convert them to the MOT format that I linked before! There is probably some code out there that would do it for you.

[cydni]

Ok thanks a lot for the help!

[danielcrane]

Ok thanks a lot for the help!

No worries at all, good luck!

[mikel-brostrom]

You may find this interesting: https://github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch

@besmaGuesmi, @cydni

[rashmi-learning]

Hii, @mikel-brostrom, I am using DeepSORT with Yolov5. I am actually figuring out the memory consumption during the tracking of multiple objects in a frame.

• System Configuration: 6 core CPU machine with 4 GB Nvidia GTX 1650 GPU.
• While running the model over the given configuration, even when I choose the device as GPU, the CPU consumption increased drastically (around 60 - 80 %) as the number of objects increased in the frame. However, the GPU consumption remains the same.

I want to run my model over GPU only without much CPU utilization. In this regard, what changes are required to run it totally over GPU with 5-15% CPU utilization only? And How to Optimize deepSORT further for low configuration devices effectively. Any help will be highly appreciated. Thank You.

[adityachintala]

Hii, @mikel-brostrom, I am using DeepSORT with Yolov5. I am actually figuring out the memory consumption during the tracking of multiple objects in a frame.

• System Configuration: 6 core CPU machine with 4 GB Nvidia GTX 1650 GPU.
• While running the model over the given configuration, even when I choose the device as GPU, the CPU consumption increased drastically (around 60 - 80 %) as the number of objects increased in the frame. However, the GPU consumption remains the same.

I want to run my model over GPU only without much CPU utilization. In this regard, what changes are required to run it totally over GPU with 5-15% CPU utilization only? And How to Optimize deepSORT further for low configuration devices effectively. Any help will be highly appreciated. Thank You.

Actually your GTX 1650 doesn't support CUDA programming https://developer.nvidia.com/cuda-gpus It can't be run on your device

[mikel-brostrom]

I want to run my model over GPU only without much CPU utilization. In this regard, what changes are required to run it totally over GPU with 5-15% CPU utilization only?

This is a bug. Solve it by adding:

import os
# limit the number of cpus used by high performance libraries
os.environ["OMP_NUM_THREADS"] = "1"
os.environ["OPENBLAS_NUM_THREADS"] = "1"
os.environ["MKL_NUM_THREADS"] = "1"
os.environ["VECLIB_MAXIMUM_THREADS"] = "1"
os.environ["NUMEXPR_NUM_THREADS"] = "1"

to the tracker file.

And How to Optimize deepSORT further for low configuration devices effectively

You can start by exporting the models to the optimal framework for your target device. Openvino for intel CPUs, NCNN for Arm chips, TFLite for Android