generate images or a video that are 360 degrees around the scene

[wmkai]

For a single scene, I want to generate images or a video that are 360 degrees around the scene. Do I need to add multiple camera rigs?

Is there an option to make the camera to complete this trajectory defined in the camera rig config? Or do you have any suggestions if I need to modify camera.py?

Thanks!

[araistrick]

Do I need to add multiple camera rigs? If you want views in multiple directions from the exact same instant in time, then add multiple camera rigs. If you want time to progress (birds fly, snakes move) as you pan the camera, then you can make it a video.

In general you can mostly avoid interacting with camera.py if you want to avoid the complexity. Just comment out our pose_cameras and animate_cameras in generate_nature.py, then use your own code to configure camera_rigs[i].location = (x,y,z); camera_rigs[i].rotation_euler=(rx,ry,rz) however you wish for each value of i.

[araistrick]

The most successful 360 images I have seen are #179 by @keithahern, I believe that issue links a fork you could try or borrow ideas from. I believe (?) the general principle of that method is to render a 360 using a single camera with very large FOV

[wmkai]

The most successful 360 images I have seen are #179 by @keithahern, I believe that issue links a fork you could try or borrow ideas from. I believe (?) the general principle of that method is to render a 360 using a single camera with very large FOV

Thank you for your reply! It seems we might not be talking about the same thing. What I want is to move and rotate the camera to render around the scene in 360 degrees and get a series of images / a series of videos / a single video, instead of a panoramic image, same as https://github.com/princeton-vl/infinigen/issues/179#issuecomment-2029102965 by @bbbbubble.

The example of images is:

And the example of video is: https://cn.dreamstime.com/%E9%87%91%E5%B1%9E%E4%B8%89%E7%BB%B4%E7%89%A9%E4%BD%93%E6%97%8B%E8%BD%AC%E7%8E%AF%E8%A7%86%E4%B8%89%E7%BB%B4%E6%B8%B2%E6%9F%93-%E9%87%91%E5%B1%9E%E5%8F%AF-d%E5%AF%B9%E8%B1%A1%E6%97%8B%E8%BD%AC%E7%8E%AF%E8%A7%86%E9%A2%91%E4%B8%BA%E9%A1%B9%E7%9B%AE%E5%9B%BE%E5%BD%A2%E8%AE%BE%E8%AE%A1%E8%89%BA%E6%9C%AF%E5%81%9A%E5%A5%BD%E5%87%86%E5%A4%87-video268380462

Update: I think my idea is the same with https://github.com/princeton-vl/infinigen/issues/211#issuecomment-2040986318

[wmkai]

Do I need to add multiple camera rigs? If you want views in multiple directions from the exact same instant in time, then add multiple camera rigs. If you want time to progress (birds fly, snakes move) as you pan the camera, then you can make it a video.

In general you can mostly avoid interacting with camera.py if you want to avoid the complexity. Just comment out our pose_cameras and animate_cameras in generate_nature.py, then use your own code to configure camera_rigs[i].location = (x,y,z); camera_rigs[i].rotation_euler=(rx,ry,rz) however you wish for each value of i.

Should I also modify camera_rigs[i].children[0].location = (x,y,z); camera_rigs[i].children[0].rotation_euler=(rx,ry,rz)? Because based on my understanding, camera_rigs[i] refers to the (i-1)-th camera rig while camera_rigs[i].children[0] refers to its 1st sub camera.

[araistrick]

If you modify the location/rotation of the camera_rig, all the cameras contained within it will also be translated/rotated automatically by blender, since their .parent is set to the camera rig. So you should not need to modify the location/rotation_euler of the children unless you want to change their offset relative to the camera rig.

[wmkai]

Do I need to add multiple camera rigs? If you want views in multiple directions from the exact same instant in time, then add multiple camera rigs. If you want time to progress (birds fly, snakes move) as you pan the camera, then you can make it a video.

In general you can mostly avoid interacting with camera.py if you want to avoid the complexity. Just comment out our pose_cameras and animate_cameras in generate_nature.py, then use your own code to configure camera_rigs[i].location = (x,y,z); camera_rigs[i].rotation_euler=(rx,ry,rz) however you wish for each value of i.

Hi, it helps me a lot. I tried this method and it worked! I set 120 camera rigs each with one camera. However, it takes 8+ hours to render a single frame when I generate high quality image in V100:

python -m infinigen.datagen.manage_jobs --output_folder outputs/dev_120cams_high_quality --num_scenes 2 \
    --pipeline_config local_256GB monocular cuda_terrain opengl_gt \
    --cleanup big_files --warmup_sec 60000 --configs high_quality_terrain \
    -p fine_terrain.mesher_backend="OcMesher"

While I only need 20 minutes to render one frame when I was generating high quality video using this command in ConfiguringInfinigen.md:

python -m infinigen.datagen.manage_jobs --output_folder outputs/my_videos --num_scenes 500 \
    --pipeline_config slurm monocular_video cuda_terrain opengl_gt \
    --cleanup big_files --warmup_sec 60000 --config trailer_video high_quality_terrain \
    -p fine_terrain.mesher_backend="OcMesher"

Any double counting involved? Why did it take so long?