"reid_hit_counter" Keeps Decreasing Causing Long-living Objects To Killed Suddenly

[h-sh-h]

Hi & Thanks for your wonderful library.

I run your tracker with "reid" feature enabled and it seems like there is a small bug in calculating "reid_hit_counter". (Although it seems too obvious so probably I am missing a point)

As the document stated :

Each tracked object keeps an internal ReID hit counter which tracks how often it's getting recognized by another tracker, each time it gets a match this counter goes up, and each time it doesn't it goes down. If it goes below 0 the object gets destroyed. If used, this argument (reid_hit_counter_max) defines how long an object can live without getting matched to any detections, before it is destroyed.

But there is no code for increasing the value of 'reid_hit_counter'. The only part of the code which I found is in tracker_step :

    def tracker_step(self):
        self.hit_counter -= 1
        if self.reid_hit_counter is None:
            if self.hit_counter <= 0:
                self.reid_hit_counter = self.reid_hit_counter_max
        else:
            self.reid_hit_counter -= 1
        self.point_hit_counter -= 1
        self.age += 1
        # Advances the tracker's state
        self.filter.predict()

So even if the object is "alive", reid_hit_counter keeps decreasing and as soon as it gets to zero, according to this section of the code :

if self.reid_hit_counter_max is None:
            self.tracked_objects = [
                o for o in self.tracked_objects if o.hit_counter_is_positive
            ]
            alive_objects = self.tracked_objects
        else:
            tracked_objects = []
            for o in self.tracked_objects:
                if o.reid_hit_counter_is_positive:
                    tracked_objects.append(o)
                    if o.hit_counter_is_positive:
                        alive_objects.append(o)
                    else:
                        dead_objects.append(o)

that object gets suddenly destroyed. It happens mostly to a long living objects in the scene which reid_hit_counter has time to get zero. Looks like the "alive" objects should not get their reid_hit_counter decreased.

Thanks in advance

[h-sh-h]

I can post you a video of this happening if it helps.

[facundo-lezama]

Hi @h-sh-h, thanks for your kind words on Norfair!

There's definitely something going on with the reid_hit_counter. I tested it myself and also saw the behavior that you mentioned. Please let me do some more work to get to the bottom of it, and I will come back to you with some new ideas.

[aguscas]

The reid_hit_counter will return to the value None as soon as the object is merged with the new uninitialized object (inside the merge method). Therefore, the way it is used is the following:

1. While the normal hit_counter is positive, the reid_hit_counter will stay as None.
2. Once the hit_counter takes the value 0, the reid_hit_counter will be set to the value reid_hit_counter_max
3. Then, each time you call the Tracker.update method, the reid_hit_counter will be decreased until the object matches with another not-yet-initialized object according to the reid_distance_function (in that case, it will be set again to None and the hit_counter will be set to a positive number, and we will be back to the case of the point 1). If reid_hit_counter reaches 0 before matching with any other object, then the corresponding object is deleted.

[h-sh-h]

Hi @aguscas I got the idea of what you are saying and it looks like norfair intended to implement the behaviour that you say but what I see in action is different.

I think there should be an additional condition of "tracked object is initialized" on setting reid_hit_counter to max when hit_counter becomes zero in tracker_step function.

I will investigate the code more tommorow

[facundo-lezama]

@h-sh-h investigating a bit further, I found that there's an issue when the objects are initialized, and tracker_step is first called since hit_counter is set to 0, and so reid_hit_counter is started.

By reordering tracker_step so that self.hit_counter -= 1 is called after addressing reid_hit_counter value, the problem is solved. You may try making the following change:

def tracker_step(self):
    if self.reid_hit_counter is None:
        if self.hit_counter <= 0:
            self.reid_hit_counter = self.reid_hit_counter_max
    else:
        self.reid_hit_counter -= 1
    self.hit_counter -= 1
    self.point_hit_counter -= 1
    self.age += 1
    # Advances the tracker's state
    self.filter.predict()

I will address this change briefly after some testing.

[facundo-lezama]

@h-sh-h we have merged a fix. You can try it out by installing Norfair from the master branch using pip install git+https://github.com/tryolabs/norfair.git@master#egg=norfair.

[h-sh-h]

@facundo-lezama Thanks for the fix. Nothing better than a cool library with bunch of fast response developers! Will certainly test that.

Currently my next steps after testing and validating my code are: 1.Implementing a norfair like tracker library in C++ optimized for embedded devices.

2. Extending norfair library to support trackers like "multi-hypothesis" ones or other features currently MATLAB MOT tracking toolbox supports.

I would be certainly happy to hear any idea that help me in my path if you have.

Bye

[facundo-lezama]

@h-sh-h glad to hear that! Those seem like great steps; I haven't gotten into "multi-hypothesis" trackers, but I will definitely take a look.

Please let us know if you find any exciting additions for Norfair, we are always open to new ideas!