Unaligned RGB and thermal frames

[NatnaelTDaba]

Hello,

Thank you for creating such a useful dataset.

My goal is to extract time-and-space aligned RGB and thermal images from their corresponding videos. By using the video splitter python script you provided in your tools repo, I was able to extract the frames as .jpeg image format. However, it seems that the extracted frames from the RGB and thermal cameras are not aligned both in time and space. This is probably due to the following reasons:

1. The recording frame rates of the RGB (10 fps) and thermal (30 fps) are not the same.
2. The duration of recording for the RGB and thermal camera are not the same.
   • E.g. for scenario 1_1_1_1, RGB video (camera_left_front) is 2 mins and 10 secs long whereas thermal video (camera_ir) is 2 mins 24 secs long
3. The provided timestamp information for both videos is not aligned.

Therefore, I wanted to know if there is a way to obtain time-and-space aligned thermal and RGB images from your dataset?

Thank you.

[rem821]

Hi,

Sorry for the late reply, but if there's anyone else that is interested about this, I'm still going to reply. The RGB and IR cameras are sadly not HW synchronized, there was no way to trigger the IR camera externally. This means that even though the timestamps provided with the video frames are accurate and synced with the rest of the device, there is no guarantee that the IR frame capture will be in sync with RGB frame capture.

Sadly I'm not very familiar with the setup for session 1&2 of the dataset (the summer part), it looks like there has been some issue with the IR camera video export causing it to stretch a little. The winter extension part of the dataset (session 3) has this issue fixed.