search

apple / ml-hypersim

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Other
1.71k stars 131 forks source link

Projection setting of the dataset #40

Closed liuzhy71 closed 3 years ago

liuzhy71 commented 3 years ago

When using the scene_annotation_tool to generate images, I found that, in some scenes, the camera projections from the scene_annotation_tool are different from the rendered images.

For example, in scene ai_026_006 the render image of cam_00 with id 0097 is frame 0097 tonemap

However, the screen shot from the annotation tool is screenshot

It can be seen that in the annotation tool, the distortion of post is small. However, in the original image the distortion is distinct.

How can I fix this problem to generate correct images from the annotation tool?

mikeroberts3000 commented 3 years ago

See #39.

liuzhy71 commented 2 years ago

camera_data_out_new.csv This is a vray camera data that I parsed and I have manually examined all the scenes for the projection alignment. Hope this helps anyone working on this problem.

mikeroberts3000 commented 2 years ago

Hi @liuzhy71, I have some good news. I just checked in some data and example code that resolves this issue.

In the contrib/mikeroberts3000 directory, I provide a modified perspective projection matrix for each scene that can be used as a drop-in replacement for the usual OpenGL perspective projection matrix, as well as example code for projecting world-space points into Hypersim images.

I apologize that it took so long to address this issue. It was especially challenging to debug because V-Ray's behavior wasn't well-documented, and I've been busy with other projects and holiday travel.

liuzhy71 commented 2 years ago

wow, thank you very much, I have made a work around of the projection matrix. But I didn't test it thoroughly, I'll check my code with yours to see if if mine is correct.