search

castacks / tartanair_tools

TartanAir dataset tools and samples
http://theairlab.org/tartanair-dataset
BSD 3-Clause "New" or "Revised" License
299 stars 34 forks source link

Optical Flow for dynamic scenes #31

Open ChenHoy opened 2 years ago

ChenHoy commented 2 years ago

Hi, I just read, that you created the optical flow from the groundtruth depth and camera motion.

However, some scenes contain dynamic objects, which apparently you mask with flow_mask.npy. Since you have a virtual dataset, I would be interested in knowing the reason for this procedure? Why could you not read out the total optical flow, resulting from all motions in the scene?

It would be a very cool feature, if you provided the total optical flow for all motions!

Amigoshan commented 2 years ago

Thanks for the question! You are right, the current optical flow can not handle dynamic objects. Due to some technical issue, we are not able to get the optical flow ground truth from the Unreal Engine. There's a recent update by the AirSim team that provides the optical flow image. But as we tested, and discussed here, it has issues with dynamic objects either, due to some issues in calculating the motion vector in Unreal Engine. For now, we only provide the optical flow mask, which can indicate where the dynamic objects are.