search

GeekAlexis / FastMOT

High-performance multiple object tracking based on YOLO, Deep SORT, and KLT 🚀
MIT License
1.16k stars 255 forks source link

How to implement cross-camera ReID #157

Open JQZhai opened 3 years ago

JQZhai commented 3 years ago

Describe what you want to do

  1. What input videos you will provide, if any: Two rtsp video streams
  2. What outputs you are expecting: Re-identify the same person in different cameras
  3. Ask your questions here, if any: How should I modify app.py

Your Environment

Any idea?Thanks.

GeekAlexis commented 3 years ago

Do your two RTSP streams have non overlapping views and the same frame rate?

JQZhai commented 3 years ago

Do your two RTSP streams have non overlapping views and the same frame rate?

Hi,GeekAlexis: yes,the two RTSP streams have non overlapping views,and their frame rates are the same.

GeekAlexis commented 3 years ago

Your case is doable but it requires a nontrivial amount of code changes. Track history needs to be shared across different MOT objects. I'll try to create a demo script for multi-camera tracking. The initial version might only support non-overlapping views. It would also help if anyone can provide sample videos to test this.

JQZhai commented 3 years ago

Thank you for your excellent work, I am looking forward to the demo script for multi-camera tracking.

weilenglxl commented 3 years ago

Hi,Alexis.I'm also looking forward to this demo, and the dataset here can be helpful: https://www.epfl.ch/labs/cvlab/data/data-pom-index-php/ https://bitbucket.org/merayxu/multiview-object-tracking-dataset/src/master/ Thanks for your excellent work again.

weilenglxl commented 3 years ago

1 Hi,GeekAlexis.If the function is like running four cameras at the same time in deepstream as shown above, how should it be added to FastMOT?I think it will be helpful for cross-camera ReID task.

weilenglxl commented 3 years ago

deepstream_multcamera.zip Please refer to this code from Deepstream with py.Thanks a lot!

GeekAlexis commented 3 years ago

I'm not familiar with the Deep Stream SDK. You might need to add an appsink in the middle of your pipeline to feed the frames into FastMOT. Related issue: #132

weilenglxl commented 3 years ago

OK, I got it,thanks. Now if I don't use Deep Stream SDK. How do I implement four rtsp streams at the same time in FastMOT?The device I use is a jetson NX that can support four network ports. At the beginning, I considered using four NX devices to run FastMOT separately, sending features to the data center server for feature matching, and then transferring the results to NXs, thus realizing cross-camera ReID, but I was not sure if it could be done in real time, and if data correlation could be achieved in one set of NX, it also solved the cost and resource overhead.

GeekAlexis commented 3 years ago

You can create 4 fastmot.VideoIO objects.

weilenglxl commented 3 years ago

Oh,that is a good idea,I will try it,thanks!

weilenglxl commented 3 years ago

Hi Alex, is it possible for me to store the features embenddings features extracted by the OsNet model in the host's database by setting up a kafka server on the host to create topics? For example mysql etc. If so where should I put the kafka code in your code? Thanks.

CompliceU commented 2 years ago

@GeekAlexis Hello, if the camera I am using has overlapping areas, I don't know how to deal with this situation better.