Closed ruffsl closed 7 years ago
Hi,
if you are using the standard detection-to-detection fusion pipeline, the detection_id in the TrackedPerson message will contain the ID of the fused detection. To find out which original detections have been fused together, you have to subscribe to the detected_persons_composite
topic, and keep a memory of those fused detections until the output from the tracker is available.
So instead of synchronizing via timestamp, you buffer the CompositeDetectedPersons until you are able to process them (maybe in a cyclic buffer so that you don't run out of memory). As the assigned (composite) detection IDs are monotonically increasing, detection IDs from different frames won't collide with each other.
I once wrote a Python module for this purpose, but I haven't tested it in a long while so I don't know if it still works. But maybe this can serve as an inspiration:
There is a usage example for the TrackSynchronizer class starting from line 172:
def callback(trackAssociation, humanAttributes):
# Do something useful with the human attributes message...
# Use trackAssociation.lookupTrackId(detectionId) to look up the track associated with a particular detection (may be None)
human_attribute_sub = message_filters.Subscriber('/spencer/perception_internal/human_attributes/rgbd_front_top', HumanAttributes)
track_sync = spencer_detected_person_association.TrackSynchronizer(human_attribute_sub, 100)
track_sync.registerCallback(callback)
Edit: I think you at least have to change the topic it subscribes to from fused_detected_persons
to detected_persons_composite
, as we renamed the message type and topic at some point.
Alright, so we've used the TrackSynchronizer and are encountering some steady state synchronization issues. Here is our current approach: main.py
We are using the TrackSynchronizer with the upper body detections then rebroadcasting the detection and track_id correspondence to a new message topic. In this file, we are using the ApproximateTimeSynchronizer to filter the RGB image, the upper body detections, the corresponding ROIs, as well as the track associations mentioned previously.
The current issues we are encountering is that the callback will proceed as expected, publishing image crops with matching track_id overlay, but only for limited time after startup; around 10 seconds. Thereafter the callback encounters issues where the list of raw detections and track associations share no common detection ids.
We hypothesize this maybe due to the fact that the phase shift in publication timestamps between the original detections message and the track association topic we create grows beyond the period of the common publication rate, as such that the ApproximateTimeFilter is aligning messages that do not truly correspond with each other.
Perhaps, you could take a look at our approach and recommend any modifications?
I'd like to be able to associate original detections from a track in order to then process those observations for classification purposes, e.g. recognise a tracked individual by appearance and assign/update the tracking ID to label or identity. In the README.md#L60, something quite equivalent to affect is aluded to:
Currently, the
spencer_tracking_msgs/TrackedPerson
message type only includes onedetection_id
field. From what I traced so far, thedetection_id
is derived from the last observation in the array of observations used during its initialisation, or updated using only the observation from the best pairing:newTrack->observation = observations.back();
trackedPerson.detection_id = track->observation->id;
Additionally, timing information from sensor input to processed output is not preserved, as detections with different modalities are fused, resulting in ill-defined timestamps to publish fused detections under. This also makes message capture/synchronization (say for post processing ROIs) using ROS topic filters via timing not as suitable. See simplified node graph for dataflow for
tracking_on_bagfile.launch
(click to expand)My question is how could we determine to what track a given observation may have been paired with? Specifically, my goal is to monitor "unlabeled" tracks, find upper body ROI pairings with those given tracks, search and recognise faces from those corresponding ROIs, then "label" those track IDs to identities as their faces come into view.
Thanks for any guidance.