Multi-object Tracking with Yolov7 & StrongSORT with OSNet

[mikel-brostrom]

Feel free to check it out :smile:

https://github.com/mikel-brostrom/Yolov7_StrongSORT_OSNet

[WongKinYiu]

Thanks.

[AlexeyAB]

@mikel-brostrom Thanks!

For some reason it fails with this error: https://colab.research.google.com/gist/AlexeyAB/fd8691990de278917f16c2ca041f13b6/yolov5-deepsort-pytorch-tutorial.ipynb

python track.py --yolo-weights yolov7.pt --img 640 --strong-sort-weights osnet_x1_0_msmt17.pt --source out.avi --save-vid

strong_sort/deep/reid/torchreid/metrics/rank.py:12: UserWarning: Cython evaluation (very fast so highly recommended) is unavailable, now use python evaluation.
  'Cython evaluation (very fast so highly recommended) is '
Fusing layers... 
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
Downloading...
From: https://drive.google.com/uc?id=1IosIFlLiulGIjwW3H8uMRmx3MzPwf86x
To: /content/Yolov7_StrongSORT_OSNet/osnet_x1_0_msmt17.pt
100% 17.3M/17.3M [00:00<00:00, 44.3MB/s]
Model: osnet_x1_0
- params: 2,193,616
- flops: 978,878,352
Successfully loaded pretrained weights from "osnet_x1_0_msmt17.pt"
** The following layers are discarded due to unmatched keys or layer size: ['classifier.weight', 'classifier.bias']
video 1/1 (1/120) /content/Yolov7_StrongSORT_OSNet/out.avi: Traceback (most recent call last):
  File "track.py", line 332, in <module>
    main(opt)
  File "track.py", line 327, in main
    run(**vars(opt))
  File "/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "track.py", line 149, in run
    for frame_idx, (path, im, im0s, vid_cap) in enumerate(dataset):
  File "/content/Yolov7_StrongSORT_OSNet/yolov7/utils/datasets.py", line 191, in __next__
    img = letterbox(img0, self.img_size, stride=self.stride)[0]
  File "/content/Yolov7_StrongSORT_OSNet/yolov7/utils/datasets.py", line 1011, in letterbox
    top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))
TypeError: type Tensor doesn't define __round__ method

[triple-Mu]

@mikel-brostrom Thanks!

For some reason it fails with this error: https://colab.research.google.com/gist/AlexeyAB/fd8691990de278917f16c2ca041f13b6/yolov5-deepsort-pytorch-tutorial.ipynb

python track.py --yolo-weights yolov7.pt --img 640 --strong-sort-weights osnet_x1_0_msmt17.pt --source out.avi --save-vid

strong_sort/deep/reid/torchreid/metrics/rank.py:12: UserWarning: Cython evaluation (very fast so highly recommended) is unavailable, now use python evaluation.
  'Cython evaluation (very fast so highly recommended) is '
Fusing layers... 
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
Downloading...
From: https://drive.google.com/uc?id=1IosIFlLiulGIjwW3H8uMRmx3MzPwf86x
To: /content/Yolov7_StrongSORT_OSNet/osnet_x1_0_msmt17.pt
100% 17.3M/17.3M [00:00<00:00, 44.3MB/s]
Model: osnet_x1_0
- params: 2,193,616
- flops: 978,878,352
Successfully loaded pretrained weights from "osnet_x1_0_msmt17.pt"
** The following layers are discarded due to unmatched keys or layer size: ['classifier.weight', 'classifier.bias']
video 1/1 (1/120) /content/Yolov7_StrongSORT_OSNet/out.avi: Traceback (most recent call last):
  File "track.py", line 332, in <module>
    main(opt)
  File "track.py", line 327, in main
    run(**vars(opt))
  File "/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "track.py", line 149, in run
    for frame_idx, (path, im, im0s, vid_cap) in enumerate(dataset):
  File "/content/Yolov7_StrongSORT_OSNet/yolov7/utils/datasets.py", line 191, in __next__
    img = letterbox(img0, self.img_size, stride=self.stride)[0]
  File "/content/Yolov7_StrongSORT_OSNet/yolov7/utils/datasets.py", line 1011, in letterbox
    top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))
TypeError: type Tensor doesn't define __round__ method

Pytorch float tensor is not a python float. Maybe we should convert dh to float(dh).

[mikel-brostrom]

Weird. It works locally for me, fails on the pipeline. No idea why it says it is a tensor. print(type(dh)) gives me <class 'numpy.float64'>. Any ideas?

[mikel-brostrom]

Maybe we should convert dh to float(dh).

Just did this and it worked :+1:

CI is now ! @AlexeyAB , @WongKinYiu, @triple-Mu

[mikel-brostrom]

updated as well!

[AlexeyAB]

@mikel-brostrom Great!

I think it makes sense to use lower confidence threshold, so it can predict better people at left top corner. conf_thres=0.15 instead of https://github.com/mikel-brostrom/Yolov7_StrongSORT_OSNet/blob/main/track.py#L52

[mikel-brostrom]

Yep @AlexeyAB , updated!

[agjunyent]

@mikel-brostrom I tried your repo on Colab. Amazing!! Is cuda available? I tried with --device 0 and it's failing

[mikel-brostrom]

If you set --device X and cuda isn't available you will get

AssertionError: CUDA unavailable, invalid device 0 requested

For activating CUDA on Colab:

Edit --> Notebook settings --> Select GPU under Hardware accelerator

[agjunyent]

@mikel-brostrom already did it, problem is that some tensors are set to CPU only, and they fail. Tried to update the code but I think it's not simply to .cpu().Tensor() the ones that fail...

[LisaChr]

@mikel-brostrom already did it, problem is that some tensors are set to CPU only, and they fail. Tried to update the code but I think it's not simply to .cpu().Tensor() the ones that fail...

@agjunyent I have the same problem. Have you been able to solve it?