search

xiexh20 / behave-dataset

Code to access BEHAVE dataset, CVPR'22
https://virtualhumans.mpi-inf.mpg.de/behave/
Other
141 stars 6 forks source link

Object Pose Coordinate System in 30 fps Dataset #31

Closed kamzero closed 11 months ago

kamzero commented 11 months ago

Hello,

I hope this message finds you well. I have been working with the 30 fps version of the dataset and have come across a discrepancy in the object pose information. As mentioned in the documentation [1], the object registration parameters are saved as axis-angle and translation in the file [obj_name]_fit.pkl. These parameters are used to transform the centered canonical templates to the Kinect camera coordinate system.

However, while examining the dataset, I noticed that for the same sequence and timestamp, the object pose stored in [obj_name]_fit.pkl and object_fit_all.npz files of the sequence Date01_Sub01_backpack_back directory do not match. This has raised some confusion regarding the coordinate system used for the object pose in the object_fit_all.npz file.

Could you please clarify the coordinate system in which the object poses are defined in the object_fit_all.npz file of the 30 fps dataset? I would greatly appreciate any insights or additional information you can provide on this matter.

Thank you for your assistance.

Best regards, kamzero

[1] Dataset documentation: https://github.com/xiexh20/behave-dataset#parse-object-pose-parameters

xiexh20 commented 11 months ago

Hi,

Thank you for your interest. We obtain the extended behave dataset (.npz files) with additional data processing, hence it is possible that the pose is slightly different from the original dataset (_fit.pkl).

More specifically, we run ICP to align the original BEHAVE object pose to the kinect point clouds, and then propagate the key frame annotations to nearby frames using ICP. Finally some smoothing is used to further improve the temporal smoothness of the object pose.

Nevertheless, the coordinate system is the same. So you can still use the same processing code mentioned in the documentation to interpret the object poses.

Hope that clarifies.

kamzero commented 11 months ago

Hi,

Thank you for your interest. We obtain the extended behave dataset (_.npz files) with additional data processing, hence it is possible that the pose is slightly different from the original dataset (__fit.pkl).

More specifically, we run ICP to align the original BEHAVE object pose to the kinect point clouds, and then propagate the key frame annotations to nearby frames using ICP. Finally some smoothing is used to further improve the temporal smoothness of the object pose.

Nevertheless, the coordinate system is the same. So you can still use the same processing code mentioned in the documentation to interpret the object poses.

Hope that clarifies.

Thanks for your reply.