How to export a generated scene to USD, FBX, or anything that can be used by Unreal or Unity?

[RodenLuo]

Hi,

I generated this video: https://i.gyazo.com/cbb7f86cdb80275d7b40176bd2a4892c.gif. The output folder looks like:

$ ls
coarse           fine_0_0_0017_0  fine_0_0_0049_0  fine_0_0_0081_0  fine_0_0_0113_0  fine_0_0_0145_0  run_pipeline.sh      savemesh_0_0_0025_0  savemesh_0_0_0057_0  savemesh_0_0_0089_0  savemesh_0_0_0121_0
fine             fine_0_0_0025_0  fine_0_0_0057_0  fine_0_0_0089_0  fine_0_0_0121_0  fine_0_0_0153_0  savemesh_0_0_0001_0  savemesh_0_0_0033_0  savemesh_0_0_0065_0  savemesh_0_0_0097_0  savemesh_0_0_0129_0
fine_0_0_0001_0  fine_0_0_0033_0  fine_0_0_0065_0  fine_0_0_0097_0  fine_0_0_0129_0  frames           savemesh_0_0_0009_0  savemesh_0_0_0041_0  savemesh_0_0_0073_0  savemesh_0_0_0105_0  savemesh_0_0_0137_0
fine_0_0_0009_0  fine_0_0_0041_0  fine_0_0_0073_0  fine_0_0_0105_0  fine_0_0_0137_0  logs             savemesh_0_0_0017_0  savemesh_0_0_0049_0  savemesh_0_0_0081_0  savemesh_0_0_0113_0  tmp

The video is generated from the frames/Image/camera_0/. I opened coarse/scene.blend and tried to export an FBX. But it contains all placeholders. In Blender, it also shows all placeholders. With "Render Image" in Blender, it indeed shows something. But it is very different from the video in the beginning, see below screenshot. How can I get a scene that is the same as in the video? Thanks!

Exported FBX shown in Houdini

[RodenLuo]

I just found https://github.com/princeton-vl/infinigen/issues/94. After I hide all placeholders and unhide all others, it's still not the same as in the video.

[araistrick]

Awesome video!

coarse/scene.blend is intentionally low poly and is used mostly for planning camera / creature motion, not for rendering.

Files of the form fine_*/scene.blend are actually used for rendering, and you should use those if you want the actual scene geometry. Apologies this isnt better documented.

Also worth noting that each fine_XXXX/scene.blend contains a different view-specific terrain marching-cubes mesh in the current Infinigen version, so there isnt a single terrain mesh that corresponds to the whole video.

[araistrick]

Another issue you will run into (even once using the fine.blend) is that the corals are instanced meshes, IE there is only a limited number of unique corals per scene and they are reused with many different positions to save cost. The FBX standard in supports exporting these, but I dont think blender's FBX exporter has implemented support for them.

Blender's USD exporter does support instances but we have experienced some issues with it in some scenes and are still working on an official solution, so it is uncharted territory to some extent.

[RodenLuo]

Hi Alex, Thanks very much! Some observations:

I was transferring files from Ubuntu (where I ran Infinigen) to Windows. I transferred the whole coarse folder and opened coarse/scene.blend in Blender on Windows. The rendering was shown in my first post.

There were the following links in the fine_ folder. I was on Termius SFTP. It complained and couldn't transfer the links. I then skipped these links and manually created a Windows shortcut for them.

~/infinigen/worldgen/outputs/my_videos/46f017a5/fine_0_0_0001_0$ ll

lrwxrwxrwx  1 luod g-luod        70 Nov 25 06:37 assets -> /home/luod/infinigen/worldgen/outputs/my_videos/46f017a5/coarse/assets/

lrwxrwxrwx  1 luod g-luod        75 Nov 25 06:43 scene.blend -> /home/luod/infinigen/worldgen/outputs/my_videos/46f017a5/coarse/scene.blend

For both of the scenes (coarse and fine), they looked missing a lot of objects. I then realized that it might be the problem of transferring. I installed a desktop on Ubuntu and opened the fine and the coarse scene natively in that Ubuntu. The rendering now makes sense. I will now experiment with a few exporting solutions and see what happens.

[RodenLuo]

Seems like only those shown in the viewport get exported. I tried to get the placeholders gone and the seaweeds and the corals back. But then the GUI is very slow. I would greatly appreciate a command line export, much like the command line rendering.

[PatchByte]

I would love infinigen to be an alternative for various different payed tier world generators such as gaea or wold creator or what ever, but I think it is not ready for that right now from what I have seen? Are there any plans to eventually being able to use infinigen for such purposes?

[araistrick]

Hello, sorry for the slow progress, we have an official exporter to USD that will be released in the next update (v1.3). I will reply to / close this issue when that is released.

[romesco]

Very excited for an official USD exporter! Thanks for all the hard work!!

[araistrick]

Hello, we've released a preview version of the export code - you can try the docs on the rc_1.3.2 branch

[bbbbubble]

Hello, we've released a preview version of the export code - you can try the docs on the rc_1.3.2 branch

@araistrick Thanks for this USD exporter. These are the outputs of the exporter, containing a .usdc file and many texture images. However, when I import these files into UE5, I can only reconstruct a very small scene, perhaps about 100m * 100m in size. I've tested many scenes and they all turn out like this. Is this correct, and how can I restore a complete large scene like in the rendered videos?

[David-Yan1]

Could you send the commands you used to run the scene and, if possible, one of the original .blend files? I'm not sure whether its something on export's end or if the original scenes were small to begin with. Thanks!

[bbbbubble]

Could you send the commands you used to run the scene and, if possible, one of the original .blend files? I'm not sure whether its something on export's end or if the original scenes were small to begin with. Thanks!

The command to generate the scene:

/root/infinigen/infinigen/datagen/manage_jobs.py --num_scenes 1 --pipeline_configs monocular_video cuda_terrain local_64GB --configs coast.gin video high_quality_terrain --pipeline_overrides iterate_scene_tasks.frame_range=[1,480]  --output_folder outputs/my_videos

The original .blend file size is 1.5 GB, so I seem unable to attach it here. However, when I open it in Blender, it only shows a small portion of the scene (this is also the case with some other scenes I've generated), just like in UE5. I suppose that in the actual InfiniGen rendering process, more than just the .blend file is used, but also peripheral scene information. Therefore, if the exporter outputs usdc, it shouldn't rely solely on the .blend file as input, it should also consider peripheral scene information, correct?