search

princeton-vl / infinigen

Infinite Photorealistic Worlds using Procedural Generation
https://infinigen.org
BSD 3-Clause "New" or "Revised" License
5.16k stars 431 forks source link

Commands to generate fish swimming around corals example scenes? #169

Closed RodenLuo closed 7 months ago

RodenLuo commented 7 months ago

Hi, Could you please kindly provide the commands to generate the (dynamic) scenes that are shown in the intro video 0:22-0:27 or similar scenes? Thanks!

araistrick commented 7 months ago

Reproducing the exact videos shown in the video is not possible due to the ongoing cross-platform-reproducibility bug, and since the exact scene you get from each random seed often changes when we modify the code.

You can however generate similar videos by following the same method we did, by running this command or a similar one, taken from our docs/ConfiguringInfinigen.md page.

python -m tools.manage_datagen_jobs --output_folder outputs/my_videos --num_scenes 500 \
    --pipeline_config slurm monocular_video cuda_terrain opengl_gt \
    --cleanup big_files --warmup_sec 60000 --config high_quality_terrain

You may need to modify it for your compute platform as described in ConfiguringInfinigen.md

This command will generate random scene types, and not all will necessarily feature fish or dynamic objects. If you want to force it to do underwater only you can add --configs under_water.gin and if you want the underwater scenes to always have fish you can add -p compose_scene.fish_school_chance=1.0.

RodenLuo commented 7 months ago

Hi Alex, Thanks very much! I should have tried the commands after I read through docs/ConfiguringInfinigen.md. Then, I will probably avoid all or most of the errors mentioned below. I put them here so that anyone facing similar errors can find this issue through the search engine. Also, a slightly better error handling might be designed. For example, if the user did not specify either slurm or local_*GB, then an error would arise. I do put a few (low-priority) questions at the end. It will be very appreciated if you can answer them at your convenience.

On Windows10 WSL2 Ubuntu 22.04.3 LTS

When I run

python -m tools.manage_datagen_jobs --output_folder outputs/my_videos --num_scenes 500     --pipeline_config slurm monocular_vide
o cuda_terrain opengl_gt     --cleanup big_files --warmup_sec 60000 --config high_quality_terrain --configs under_water.gin -p compose_scene.fish_school_chance=1.0 -p compose_scene.
corals_chance=1.0

I got 

WARNING:root:SMB_AUTH envvar is not set, smb_client upload will not work. Ignore this message if not using upload                                                                    
Using get_slurm_banned_nodes()=[]                                                                                                                                                    
outputs/my_videos 11/13 08:43AM -> 11/13 08:43AM                                                                                                                                     
Traceback (most recent call last):                                                                                                                                                   
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/runpy.py", line 196, in _run_module_as_main                                                                             
    return _run_code(code, main_globals, None,                                                                                                                                       
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/runpy.py", line 86, in _run_code                                                                                        
    exec(code, run_globals)                                                                                                                                                          
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 826, in <module>                                                                                          
    main(args)                                                                                                                                                                       
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                                                                                                                          
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                                                                                                                    
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)                                                                                                                                               
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 655, in main                                                                                              
    manage_datagen_jobs(all_scenes, elapsed=(now-start_time).total_seconds())                                                                                                        
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                                                                                                                          
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                                                                                                                    
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)                                                                                                                                               
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 594, in manage_datagen_jobs                                                                               
    control_state = compute_control_state(args, totals, elapsed, num_concurrent)                                                                                                     
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 556, in compute_control_state                                                                             
    num_concurrent = int(os.environ[NUM_CONCURRENT_ENVVAR])                                                                                                                          
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/os.py", line 680, in __getitem__                                                                                        
    raise KeyError(key) from None                                                                                                                                                    
KeyError: 'INFINIGEN_NUMCONCURRENT_TARGET'                                                                                                                                           
  In call to configurable 'manage_datagen_jobs' (<function manage_datagen_jobs at 0x7fc05cdc2830>)                                                                                   
  In call to configurable 'main' (<function main at 0x7fc05cdc1c60>)

Setting NUM_CONCURRENT_ENVVAR=32 or NUM_CONCURRENT_ENVVAR=32 in bash terminal doesn't help.

Then I removed the slurm from --pipeline_config and run 

python -m tools.manage_datagen_jobs --output_folder outputs/my_videos --num_scenes 500     --pipeline_config monocular_video cuda
_terrain opengl_gt     --cleanup big_files --warmup_sec 60000 --config high_quality_terrain --configs under_water.gin -p compose_scene.fish_school_chance=1.0 -p compose_scene.corals
_chance=1.0

I got 

WARNING:root:SMB_AUTH envvar is not set, smb_client upload will not work. Ignore this message if not using upload                                                                    
Using get_slurm_banned_nodes()=[]                                                                                                                                                    
outputs/my_videos 11/13 09:56AM -> 11/13 09:56AM                                                                                                                                     
Traceback (most recent call last):                                                                                                                                                   
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/runpy.py", line 196, in _run_module_as_main                                                                             
    return _run_code(code, main_globals, None,                                                                                                                                       
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/runpy.py", line 86, in _run_code                                                                                        
    exec(code, run_globals)                                                                                                                                                          
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 826, in <module>                                                                                          
    main(args)                                                                                                                                                                       
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                               
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                         
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)       
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 655, in main   
    manage_datagen_jobs(all_scenes, elapsed=(now-start_time).total_seconds())             
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                               
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                         
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)       
TypeError: manage_datagen_jobs() missing 1 required positional argument: 'num_concurrent'                                                                                            
  No values supplied by Gin or caller for arguments: ['num_concurrent']                   
  Gin had values bound for: []               
  Caller supplied values for: ['all_scenes', 'elapsed']                                   
  In call to configurable 'manage_datagen_jobs' (<function manage_datagen_jobs at 0x7fa49cbeaa70>)                                                                                   
  In call to configurable 'main' (<function main at 0x7fa49cbe9e10>)

Then I added --num_concurrent 5 and got 

python -m tools.manage_datagen_jobs --output_folder outputs/my_videos --num_scenes 500 --num_concurrent 5  --pipeline_config mono
cular_video cuda_terrain opengl_gt     --cleanup big_files --warmup_sec 60000 --config high_quality_terrain --configs under_water.gin -p compose_scene.fish_school_chance=1.0 -p comp
ose_scene.corals_chance=1.0                                                                                                                                                          
WARNING:root:SMB_AUTH envvar is not set, smb_client upload will not work. Ignore this message if not using upload                                                                    
usage: manage_datagen_jobs.py [-h] -o OUTPUT_FOLDER [--blender_path BLENDER_PATH] [--num_scenes NUM_SCENES] [--meta_seed META_SEED]                                                  
                              [--specific_seed SPECIFIC_SEED [SPECIFIC_SEED ...]] [--use_existing] [--warmup_sec WARMUP_SEC]                                                         
                              [--cleanup {all,big_files,none,except_logs,except_crashed}] [--configs [CONFIGS ...]] [-p OVERRIDES [OVERRIDES ...]]                                   
                              [--wandb_mode {online,offline,disabled}] [--pipeline_configs PIPELINE_CONFIGS [PIPELINE_CONFIGS ...]]                                                  
                              [--pipeline_overrides PIPELINE_OVERRIDES [PIPELINE_OVERRIDES ...]] [--overwrite] [-d] [-v]                                                             
manage_datagen_jobs.py: error: unrecognized arguments: --num_concurrent 5

Switched to --pipeline_overrides manage_datagen_jobs.num_concurrent=5 and got 

WARNING:root:SMB_AUTH envvar is not set, smb_client upload will not work. Ignore this message if not using upload                                                                    
Using get_slurm_banned_nodes()=[]                                                                                                                                                    
outputs/my_videos 11/13 10:06AM -> 11/13 10:06AM                                                                                                                                     
============================================================                                                                                                                         
control_state/curr_concurrent_max : 1                                                                                                                                                
control_state/disk_usage       : 0.18                                                                                                                                                
control_state/n_in_flight      : 0                                                                                                                                                   
control_state/try_to_launch    : 1                                                                                                                                                   
control_state/will_launch      : 1                                                                                                                                                   
------------------------------------------------------------                                                                                                                         
Traceback (most recent call last):                                                                                                                                                   
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/runpy.py", line 196, in _run_module_as_main                                                                             
    return _run_code(code, main_globals, None,                                                                                                                                       
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/runpy.py", line 86, in _run_code                                                                                        
    exec(code, run_globals)                                                                                                                                                          
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 826, in <module>                                                                                          
    main(args)                                                                                                                                                                       
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                                                                                                                          
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                         
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)       
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 655, in main   
    manage_datagen_jobs(all_scenes, elapsed=(now-start_time).total_seconds())             
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                               
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                         
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)       
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 613, in manage_datagen_jobs                                                                               
    run_task(queue_func, args.output_folder / str(scene['seed']), scene, taskname)        
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                               
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise                                          
    raise proxy.with_traceback(exception.__traceback__) from None                         
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)       
  File "/home/roden/infinigen/worldgen/tools/manage_datagen_jobs.py", line 326, in run_task                                                                                          
    job_obj, output_folder = queue_func(     
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1605, in gin_wrapper                                                                 
    utils.augment_exception_message_and_reraise(e, err_str)                               
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None                         
  File "/home/roden/miniconda3/envs/infinigen/lib/python3.10/site-packages/gin/config.py", line 1582, in gin_wrapper                                                                 
    return fn(*new_args, **new_kwargs)       
TypeError: queue_coarse() missing 1 required positional argument: 'submit_cmd'            
  No values supplied by Gin or caller for arguments: ['submit_cmd']                       
  Gin had values bound for: []               
  Caller supplied values for: ['configs', 'folder', 'input_indices', 'name', 'output_indices', 'overrides', 'seed', 'taskname']                                                      
  In call to configurable 'queue_coarse' (<function queue_coarse at 0x7fe3e54525f0>)      
  In call to configurable 'run_task' (<function run_task at 0x7fe3e546e830>)              
  In call to configurable 'manage_datagen_jobs' (<function manage_datagen_jobs at 0x7fe3e546ea70>)                                                                                   
  In call to configurable 'main' (<function main at 0x7fe3e546de10>)

submit_cmd gives me the feeling it should be handled internally. So I stopped here...

Then I ran the automatic hello world example, and it ran through. 

python -m tools.manage_datagen_jobs --output_folder outputs/hello_world_job --num_scenes 1 --specific_seed 0 \
--configs desert.gin simple.gin --pipeline_configs local_16GB.gin monocular.gin blender_gt.gin --pipeline_overrides LocalScheduleHandler.use_gpu=True
WARNING:root:SMB_AUTH envvar is not set, smb_client upload will not work. Ignore this message if not using upload
Using get_slurm_banned_nodes()=[]
outputs/hello_world_job 11/13 10:58AM -> 11/13 10:58AM
============================================================
control_state/curr_concurrent_max : 1
control_state/disk_usage       : 0.18
control_state/n_in_flight      : 0
control_state/try_to_launch    : 1
control_state/will_launch      : 1
------------------------------------------------------------
outputs/hello_world_job 11/13 10:58AM -> 11/13 10:58AM
============================================================
control_state/curr_concurrent_max : 1
control_state/disk_usage       : 0.18
control_state/n_in_flight      : 1
control_state/try_to_launch    : 0
control_state/will_launch      : 0
running/coarse                 : 1
running/total                  : 1
------------------------------------------------------------
outputs/hello_world_job 11/13 10:58AM -> 11/13 10:58AM
......
(Keeps logging like the above)

Then, I compared the two commands and realized the subtle difference of the s behind --config and --pipeline_config as well as the existence of local_16GB in the hello world example. Then I tried 

python -m tools.manage_datagen_jobs --output_folder outputs/my_videos --num_scenes 500  --pipeline_config monocular_video cuda_te
rrain opengl_gt local_64GB  --cleanup big_files --warmup_sec 60000 --configs under_water.gin high_quality_terrain.gin -p compose_scene.fish_school_chance=1.0 -p compose_scene.corals
_chance=1.0

And it finally ran through.

Some questions:

  1. WARNING:root:SMB_AUTH envvar is not set, smb_client upload will not work. Ignore this message if not using upload

    • This warning is raised from all commands. Does it matter?

  2. Using get_slurm_banned_nodes()=[]

    • This is from the automatic hello world command. I did not specify anything about slurm. Why would it be logged, and should I care about it?
  3. For the subtle difference of the s behind --config and --pipeline_config, it gives me the feeling that without s, one can remove .gin. Is this all the design choice here, or are there other considerations?
araistrick commented 7 months ago

Thanks for following up!

RE questions 1&2, these warnings are not problematic for you, at some point ill remove them for non-slurm users.

RE 3, I believe the question whether/why we do/do not pluralize --config vs --configs or --pipeline_config vs --pipeline_configs. This is not deliberate and will be standardized, I think (?) I may have already done this on our rc_1.1.1 release candidate branch but I do not remember exactly. Sorry for the confusion regardless.