Is it possible to get the complete 3D mesh (including objects), and light sources information of the scenes?

[ttsesm]

Hi,

Thank you for the dataset. I would like to ask though whether it would be possible to get/extract the complete ground truth 3D mesh with the objects, as well as the light sources properties (e.g. position, dimensions, possibly IES data) used within the scenes.

Thanks.

[bertjiazheng]

Thanks for your interest in our dataset. However, we could not provide the CAD models of the objects, and mesh models of the scene due to the copyright issue.

[ttsesm]

Oh, I see. So getting the the 3d mesh through the visualize_mesh.py is the only option.

I've tried to run the script though but I am getting the following error in line https://github.com/bertjiazheng/Structured3D/blob/c5eb9a85ae39e6a238a1034cc7e2453d8167e0e5/visualize_mesh.py#L123

Traceback (most recent call last):
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/pydevd.py", line 1438, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 262, in <module>
    main()
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 258, in main
    visualize_mesh(args)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 235, in visualize_mesh
    mesh = create_plane_mesh(corners, corner_floor, textures, texture_floor, texture_ceiling,
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 123, in create_plane_mesh
    mesh.texture = open3d.geometry.Image(textures)
AttributeError: 'open3d.open3d_pybind.geometry.TriangleMesh' object has no attribute 'texture'

It seems that in the latest open3d version the attribute texture is replaced with textures. However, if I make the change I am getting the following error:

Traceback (most recent call last):
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/pydevd.py", line 1438, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 262, in <module>
    main()
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 258, in main
    visualize_mesh(args)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 235, in visualize_mesh
    mesh = create_plane_mesh(corners, corner_floor, textures, texture_floor, texture_ceiling,
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 123, in create_plane_mesh
    mesh.textures = open3d.geometry.Image(textures)
TypeError: (): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.open3d_pybind.geometry.TriangleMesh, arg0: List[open3d::geometry::Image]) -> None
Invoked with: geometry::TriangleMesh with 24 points and 12 triangles., Image of size 3072x512, with 3 channels.
Use numpy.asarray to access buffer data.

Which I am trying to figure out what is the cause. If you have any idea how to resolve this, please feel free to comment.

[bertjiazheng]

Please use the 0.9 version of the open3d:
pip install open3d==0.9.0.0

The _visualizemesh provides a simple visualization of the 3D layout as shown in the README.

[ttsesm]

The _visualizemesh provides a simple visualization of the 3D layout as shown in the README.

I see, so not depth information of the objects within. Is it possible to obtain the 3D layout based on the panorama rgbd input or any other way?

[bertjiazheng]

What do you mean by the 3D layout?

[ttsesm]

Apologies my bad, I mean to reconstruct the 3D mesh from the panorama rgbd images.

[bertjiazheng]

There are only a few images (i.e., one panorama per room, and up to 5 perspective images per room) provided in the scene. So it may not get satisfying reconstruction results from a few images.

[ttsesm]

Yup this will not be helpful for me. I would be interested either for the ground truth 3D layouts with objects and material properties or at least a good 3D mesh reconstruction with the albedo maps of the scenes.

Anyways, thank you for your time.