bertjiazheng / Structured3D

[ECCV'20] Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling
https://structured3d-dataset.org
MIT License
534 stars 62 forks source link

Is it possible to get the complete 3D mesh (including objects), and light sources information of the scenes? #15

Closed ttsesm closed 4 years ago

ttsesm commented 4 years ago

Hi,

Thank you for the dataset. I would like to ask though whether it would be possible to get/extract the complete ground truth 3D mesh with the objects, as well as the light sources properties (e.g. position, dimensions, possibly IES data) used within the scenes.

Thanks.

bertjiazheng commented 4 years ago

Thanks for your interest in our dataset. However, we could not provide the CAD models of the objects, and mesh models of the scene due to the copyright issue.

ttsesm commented 4 years ago

Oh, I see. So getting the the 3d mesh through the visualize_mesh.py is the only option.

I've tried to run the script though but I am getting the following error in line https://github.com/bertjiazheng/Structured3D/blob/c5eb9a85ae39e6a238a1034cc7e2453d8167e0e5/visualize_mesh.py#L123

Traceback (most recent call last):
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/pydevd.py", line 1438, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 262, in <module>
    main()
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 258, in main
    visualize_mesh(args)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 235, in visualize_mesh
    mesh = create_plane_mesh(corners, corner_floor, textures, texture_floor, texture_ceiling,
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 123, in create_plane_mesh
    mesh.texture = open3d.geometry.Image(textures)
AttributeError: 'open3d.open3d_pybind.geometry.TriangleMesh' object has no attribute 'texture'

It seems that in the latest open3d version the attribute texture is replaced with textures. However, if I make the change I am getting the following error:

Traceback (most recent call last):
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/pydevd.py", line 1438, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/usr/share/pycharm/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 262, in <module>
    main()
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 258, in main
    visualize_mesh(args)
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 235, in visualize_mesh
    mesh = create_plane_mesh(corners, corner_floor, textures, texture_floor, texture_ceiling,
  File "/home/ttsesm/Development/Structured3D/visualize_mesh.py", line 123, in create_plane_mesh
    mesh.textures = open3d.geometry.Image(textures)
TypeError: (): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.open3d_pybind.geometry.TriangleMesh, arg0: List[open3d::geometry::Image]) -> None
Invoked with: geometry::TriangleMesh with 24 points and 12 triangles., Image of size 3072x512, with 3 channels.
Use numpy.asarray to access buffer data.

Which I am trying to figure out what is the cause. If you have any idea how to resolve this, please feel free to comment.

bertjiazheng commented 4 years ago

Please use the 0.9 version of the open3d:
pip install open3d==0.9.0.0

The _visualizemesh provides a simple visualization of the 3D layout as shown in the README.

ttsesm commented 4 years ago

The _visualizemesh provides a simple visualization of the 3D layout as shown in the README.

I see, so not depth information of the objects within. Is it possible to obtain the 3D layout based on the panorama rgbd input or any other way?

bertjiazheng commented 4 years ago

What do you mean by the 3D layout?

ttsesm commented 4 years ago

Apologies my bad, I mean to reconstruct the 3D mesh from the panorama rgbd images.

bertjiazheng commented 4 years ago

There are only a few images (i.e., one panorama per room, and up to 5 perspective images per room) provided in the scene. So it may not get satisfying reconstruction results from a few images.

ttsesm commented 4 years ago

Yup this will not be helpful for me. I would be interested either for the ground truth 3D layouts with objects and material properties or at least a good 3D mesh reconstruction with the albedo maps of the scenes.

Anyways, thank you for your time.