Object class do not change when tracking

[MichaelLiLee]

Hi Jason

I am using your repository to implement a human object interaction (HOI) detection, but i meet a problem.

problem description: a video contain a person who use a spatula to flip the dough, when dough is cooked , it become a pancake. my objective is to detect and track both dough and pancake. here is the experience result i put on youtube. as you can see, object class do not change when tracking(dough to pancake).

how can i fix this problem?could you do me a favor to help me?

[tensorturtle]

Hi, thanks for your feedback and video demo. I fully understand the issue you are having, and I know why. Essentially, the KalmanBoxTracker object (which represents a tracked bounding box) is initiated with the bounding box's category at the beginning of the track, but there is no update mechanism.

I plan on fixing it by passing in the category in the update() method for KalmanBoxTracker and updating self.detclass attribute in https://github.com/tensorturtle/classy-sort-yolov5/blob/main/sort/sort.py

I will work on it and get back to you.

[tensorturtle]

@MichaelLiLee I made a commit that should fix the issue. Please try it out and tell me if it helped.

[MichaelLiLee]

Hi Jason I am very thankful that you reply me quickly.

I have tried your new commit, and now it can change object category properly. here is the video record.

But i have another problem, would it be possible to assign a new id when object category was changed(for this scenario: id 8 become id 9 when dough became pancake).

[tensorturtle]

I'm glad it worked.

Here is the fix for your second problem:

Go to sort/sort.py, line 138 At the end of the update() method, add this code:

if self.detclass != bbox[5]:
    self.detclass = bbox[5]
    self.id = KalmanBoxTracker.count
    KalmanBoxTracker.count += 1

Because this is a specific feature and not generally applicable, I will not be including this as part of the official code. Also, I haven't tested this so you will need to try running it yourself.

[MichaelLiLee]

I have tested your code on the video, and It works exactly as expected. Thank you for your kind assistance.

[tensorturtle]

@MichaelLiLee Happy to help. Good luck with your project