Large sections of dropped messages

[dwisth]

Hi @hemi86,

First of all, thank for you for the challenge!

I was processing LAB_Survey_2.bag and noticed there was large gaps in recorded messages in the rosbag.

For example, when I run rqt_bag LAB_Survey_2.bag I see the following output: zooming in.. zooming in more..

You can see from the images above that there are many gaps of ~1 second length where almost no data is recorded. From our experience, this usually happens when we try to more data than the disk can handle so messages get dropped.

This can make the rosbag very hard to process because all sensor signals are missing during these times, even the IMU.

I haven't checked the other rosbags in detail (some of them seem okay) but it's very obvious in LAB_Survey_2.bag.

[hemi86]

Hi @dwisth, the times that you see in the rqt_bag are the times of arrival of the individual images, not the time when they were captured (this timestamp is in the header). We are currently checking if we lost any frames, or if this is just a mismatch between time-of-arrival vs time captured.

[bedaberner]

The bag looks really bad when you look at time of arrival. However if you rewrite the rosbag with the timestamps, it's much better!: There actually are some parts where the imu_adis drops messages but the other topics are fine.

Time of arrival is really meaningless in this context, if you want the rosbags to also publish the messages according to the header timestamps, you can use this script to rewrite them:

import rosbag

with rosbag.Bag('output.bag', 'w') as outbag:
    for topic, msg, t in rosbag.Bag('input.bag').read_messages():
        # This also replaces tf timestamps under the assumption 
        # that all transforms in the message share the same timestamp
        if topic == "/tf" and msg.transforms:
            outbag.write(topic, msg, msg.transforms[0].header.stamp)
        else:
            outbag.write(topic, msg, msg.header.stamp if msg._has_header else t)

Source

[dwisth]

@bedaberner - I just tested out your script - there's still a couple of missing frames but I think it's much better now. I'll get back into getting results for the competition now.

Thanks!