search

DLR-RM / BlenderProc

A procedural Blender pipeline for photorealistic training image generation
GNU General Public License v3.0
2.82k stars 452 forks source link

Support for OpenRooms dataset #214

Closed ttsesm closed 3 years ago

ttsesm commented 3 years ago

Hi guys,

Is there any plan for supporting the OpenRooms dataset through BlenderProc?

Thanks.

themasterlink commented 3 years ago

Hey,

It seems that this is only an image based dataset, where they used objects from here:

Related Datasets
The OpenRooms dataset is built on the datasets listed below. We thank their creators for the excellent contribution. Please refer to prior datasets for license issues and terms of use if you hope to use them to create your own dataset.

ScanNet dataset: The real 3D scans of indoor scenes.
Scan2cad dataset: The alignment of CAD models to the scanned point clouds.
Laval outdoor lighting dataset: HDR outdoor environment maps
HDRI Haven lighting dataset: HDR outdoor environment maps
PartNet dataset: CAD models

So the question would be what exactly we should provide then?

Best, Max

ttsesm commented 3 years ago

Hi Max,

Indeed in this first version of the dataset they provide only the images but soon they claim that they will make available the rest of the tools, hopefully. Also based on the following description:

... We demonstrate an instantiation of our approach on the publicly available ScanNet dataset. Deep networks trained on our proposed dataset achieve competitive performance for shape, material and lighting estimation on real images and can be used for photorealistic augmented reality applications, such as object insertion and material editing. Importantly, the dataset and all the tools to create such datasets from scans will be released, enabling others in the community to easily build large-scale datasets of their own.

as well as in the following video:

http://cseweb.ucsd.edu/~viscomp/projects/CVPR21OpenRooms/video.mp4

These tools might be useful for creating rooms (CAD models), as well as other features, e.g. lighting, materials, furniture position, etc. in the room. Thus, I was thinking that it might be interesting for something similar to the random room constructor module.

/Theo

themasterlink commented 3 years ago

Hey,

as soon as they have a 3D dataset with publicly available models, we will find a way to support it :)

But before they release that we can't really do much :/

Best, Max