Export as Pointcloud or standard Mesh Formats

princeton-vl / infinigen

Infinite Photorealistic Worlds using Procedural Generation

https://infinigen.org

BSD 3-Clause "New" or "Revised" License

5.16k stars 431 forks source link

Export as Pointcloud or standard Mesh Formats #198

Open enor2017 opened 4 months ago

enor2017 commented 4 months ago

Hi, I am extremely impressed by this project, and thank you for your contribution. Since this is a 3D generation model, I am wondering whether I could export the generated scene (with all objects instanciated, no placeholders) to a commonly-used point cloud format (such as ply/npz) or mesh format (such as obj/ply)? I have read through the issues, and here is my progress:

It seems possible to unhide the viewports using blender, but when I export the scene as obj, I still can only see bounding boxes (placeholders), any step I missed? Moreover, can this process be automatic?
I have checked the save_mesh task, but the generated mesh is a customized format in .npz file. Is there any instruction on how I could convert this file into a point cloud easily?
I noticed that exporting as .ply was added to the (internal?) TODO list several months ago, any updates or simple instructions on this?

Thank you so much for your reply!

araistrick commented 3 months ago

Hello, we are working on this and it will be released soon.

It is worth noting however that it is simply impossible to export to a single fully realized point cloud at full resolution. The only way to represent infinigen scenes at the full resolution shown in images is by not duplicating redundant information, IE remembering that there is only 10 unique rocks or 10 unique grass tufts, and storing only the poses of these unique assets, not many duplicated copies. The only mesh formats that support this are .blend, .usd or a custom python solution. Our exporter will support .usd when it is released, ideally within the next two weeks. You can also attempt to use blenders usd exporter in the meantime.

enor2017 commented 3 months ago

Hello, we are working on this and it will be released soon.

It is worth noting however that it is simply impossible to export to a single fully realized point cloud at full resolution. The only way to represent infinigen scenes at the full resolution shown in images is by not duplicating redundant information, IE remembering that there is only 10 unique rocks or 10 unique grass tufts, and storing only the poses of these unique assets, not many duplicated copies. The only mesh formats that support this are .blend, .usd or a custom python solution. Our exporter will support .usd when it is released, ideally within the next two weeks. You can also attempt to use blenders usd exporter in the meantime.

Thank you for your reply! I understand the scene is too large to export fully. Is there any method for me to fully export only a small region of the instantiated scene? Like crop a HxWxD region and export it.

araistrick commented 3 months ago

Cropping a region is be possible but would take nontrivial code, specifically to loop through the instances in the depsgraph and check which ones are in the region.

The easiest solution would be to just generate a scene of size HxWxD then export the whole thing. @mazeyu how does one control scene size? we should make this an easily accessible option

mazeyu commented 3 months ago

Currently only the dimension of the populated terrain is controlled. I will write an option to control both the populated terrain and final terrain and let you know.

enor2017 commented 3 months ago

Thanks @araistrick @mazeyu, and I look forward to your wonderful work!

andytriboletti commented 2 months ago

I'm looking forward to the export mesh usd solution. Would like to include these scenes in a game.

araistrick commented 2 months ago

@mazeyu I believe we've published the scene-size control as part of v1.2.5 now, could you share the code snippet for it?

mazeyu commented 2 months ago

https://github.com/princeton-vl/infinigen/blob/main/infinigen/terrain/core.py#L90 This line and the next line control the bounds for non-terrain assets and bounds for the terrain. (use gin to modify)