How to use the dataset's transforms?

[QiukuZ]

Thank you very much for your excellent work! While use your dataset, how can i get every image's camera to world pose?

How do I apply the transform I get from the .json？ Do I need to convert from blender coordinate system to opencv coordinate system like nerf?

[thomasneff]

Hi!

You can see how we use our transforms in the following code portions:

1.) Here we generate ray directions for a given image width, height, field of view and focal depth:

https://github.com/facebookresearch/DONERF/blob/main/src/datasets.py#L234 https://github.com/facebookresearch/DONERF/blob/main/src/util/raygeneration.py

2.) From these directions, we use the rotation matrix that we get from the transforms .json to orient the rays correctly:

https://github.com/facebookresearch/DONERF/blob/main/src/features.py#L376 https://github.com/facebookresearch/DONERF/blob/main/src/nerf_raymarch_common.py#L59

3.) The rotation matrix and the world-space position is computed here (there is another dataloader, but it does the same thing): https://github.com/facebookresearch/DONERF/blob/main/src/datasets.py#L229

In general, as long as your usage of coordinate spaces stays consistent throughout, it does not really matter which coordinate system you use. For our dataset, we directly export them from Blender (similar to what the original NeRF datasets did). If you need to interface with datasets that use different modalities, you might need to rotate/invert some of the axes of the transform that you get in our .json files.

To be more specific: For our dataset, we directly export matrix_world from the camera object that we have in Blender, i.e., we export the rotation and translation of the camera.

[shingkim]

good work, But i not clear what is the content of the dataset_info.json file, and how to get it?