Synchronization issue between raw detections and track associations

[jaskaran1]

This is with reference to #34

Alright, so we've used the TrackSynchronizer and are encountering some steady state synchronization issues. Here is our current approach: main.py

We are using the TrackSynchronizer with the upper body detections then rebroadcasting the detection and track_id correspondence to a new message topic. In this file, we are using the ApproximateTimeSynchronizer to filter the RGB image, the upper body detections, the corresponding ROIs, as well as the track associations mentioned previously.

The current issues we are encountering is that the callback will proceed as expected, publishing image crops with matching track_id overlay, but only for limited time after startup; around 10 seconds. Thereafter the callback encounters issues where the list of raw detections and track associations share no common detection ids.

We hypothesize this maybe due to the fact that the phase shift in publication timestamps between the original detections message and the track association topic we create grows beyond the period of the common publication rate, as such that the ApproximateTimeFilter is aligning messages that do not truly correspond with each other.

Perhaps, you could take a look at our approach and recommend any modifications?

[tlind]

[...] as such that the ApproximateTimeFilter is aligning messages that do not truly correspond with each other.

If I interpret your code correctly, you are using a sync slop of 20 ms for the ApproximateTimeSynchronizer. Then your statement doesn't make much sense, because the synchronizer should discard any message tuples which are out of sync by more than that amount of time (i.e. callback shouldn't even get called).

I know that the Python implementation of the ApproximateTimeSynchronizer doesn't perform nearly as well as the C++ counterpart. My experience is that with such long message queues (you are using queue_size 300), the synchronization can stall completely. We had to reduce the queue size to below 50 for the synchronization between tracks and groundtruth to work in our ICRA 16 paper. Not sure why, maybe there is something in the Python code that scales quadratically with the number of messages in the synchronizer queue.

You also have very long publisher queues. Maybe reducing the queue sizes altogether could help, but I am just guessing wildly now.