search

ayaanzhaque / instruct-nerf2nerf

Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions (ICCV 2023)
https://instruct-nerf2nerf.github.io/
MIT License
769 stars 64 forks source link

invalid literal for int() with base 10: 'config' #7

Closed samsullivandelgobbo closed 1 year ago

samsullivandelgobbo commented 1 year ago

System:

Windows 10 Version 10.0.19044 Build 190 Ubuntu 20.04 running in WSL2 CUDA 11.7 installed using miniconda in ~/miniconda3/envs/nerfstudio/ Torch 1.13.1+cu117

Problem:

Trying to train instruct2nerf model on an freshly trained and processed model using the following command ns-train in2n --data outputs/office --load-dir outputs/office/nerfacto/2023-04-12_102834 --pipeline.prompt 'make the floor lava' --pipeline.guidance-scale 7.5 --pipeline.image-guidance-scale 1.5

fails and returns ValueError: invalid literal for int() with base 10: 'config'

I can't seem to fix it, tried reinstalling both using the basic pip command and through directly cloning the repo. Seems like the config file is being read incorrectly. Has anyone run into this?

Full log:

Setting up training dataset...
Caching all 239 images.
Setting up evaluation dataset...
Caching all 26 images.
Using full precision
InstructPix2Pix loaded!
Loading latest checkpoint from load_dir
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/bouba/miniconda3/envs/nerfstudio/bin/ns-train:8 in <module>                                │
│                                                                                                  │
│   5 from scripts.train import entrypoint                                                         │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(entrypoint())                                                                   │
│   9                                                                                              │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/scripts/train.py:247 in       │
│ entrypoint                                                                                       │
│                                                                                                  │
│   244 │   """Entrypoint for use with pyproject scripts."""                                       │
│   245 │   # Choose a base configuration and override values.                                     │
│   246 │   tyro.extras.set_accent_color("bright_yellow")                                          │
│ ❱ 247 │   main(                                                                                  │
│   248 │   │   tyro.cli(                                                                          │
│   249 │   │   │   AnnotatedBaseConfigUnion,                                                      │
│   250 │   │   │   description=convert_markup_to_ansi(__doc__),                                   │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/scripts/train.py:233 in main  │
│                                                                                                  │
│   230 │   config.print_to_terminal()                                                             │
│   231 │   config.save_config()                                                                   │
│   232 │                                                                                          │
│ ❱ 233 │   launch(                                                                                │
│   234 │   │   main_func=train_loop,                                                              │
│   235 │   │   num_gpus_per_machine=config.machine.num_gpus,                                      │
│   236 │   │   num_machines=config.machine.num_machines,                                          │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/scripts/train.py:172 in       │
│ launch                                                                                           │
│                                                                                                  │
│   169 │   │   # world_size=0 uses one CPU in one process.                                        │
│   170 │   │   # world_size=1 uses one GPU in one process.                                        │
│   171 │   │   try:                                                                               │
│ ❱ 172 │   │   │   main_func(local_rank=0, world_size=world_size, config=config)                  │
│   173 │   │   except KeyboardInterrupt:                                                          │
│   174 │   │   │   # print the stack trace                                                        │
│   175 │   │   │   CONSOLE.print(traceback.format_exc())                                          │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/scripts/train.py:86 in        │
│ train_loop                                                                                       │
│                                                                                                  │
│    83 │   """                                                                                    │
│    84 │   _set_random_seed(config.machine.seed + global_rank)                                    │
│    85 │   trainer = config.setup(local_rank=local_rank, world_size=world_size)                   │
│ ❱  86 │   trainer.setup()                                                                        │
│    87 │   trainer.train()                                                                        │
│    88                                                                                            │
│    89                                                                                            │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/engine/trainer.py: │
│ 146 in setup                                                                                     │
│                                                                                                  │
│   143 │   │   )                                                                                  │
│   144 │   │   self.optimizers = self.setup_optimizers()                                          │
│   145 │   │                                                                                      │
│ ❱ 146 │   │   self._load_checkpoint()                                                            │
│   147 │   │                                                                                      │
│   148 │   │   self.callbacks = self.pipeline.get_training_callbacks(                             │
│   149 │   │   │   TrainingCallbackAttributes(                                                    │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/engine/trainer.py: │
│ 339 in _load_checkpoint                                                                          │
│                                                                                                  │
│   336 │   │   │   if load_step is None:                                                          │
│   337 │   │   │   │   print("Loading latest checkpoint from load_dir")                           │
│   338 │   │   │   │   # NOTE: this is specific to the checkpoint name format                     │
│ ❱ 339 │   │   │   │   load_step = sorted(int(x[x.find("-") + 1 : x.find(".")]) for x in os.lis   │
│   340 │   │   │   load_path: Path = load_dir / f"step-{load_step:09d}.ckpt"                      │
│   341 │   │   │   assert load_path.exists(), f"Checkpoint {load_path} does not exist"            │
│   342 │   │   │   loaded_state = torch.load(load_path, map_location="cpu")                       │
│                                                                                                  │
│ /home/bouba/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/engine/trainer.py: │
│ 339 in <genexpr>                                                                                 │
│                                                                                                  │
│   336 │   │   │   if load_step is None:                                                          │
│   337 │   │   │   │   print("Loading latest checkpoint from load_dir")                           │
│   338 │   │   │   │   # NOTE: this is specific to the checkpoint name format                     │
│ ❱ 339 │   │   │   │   load_step = sorted(int(x[x.find("-") + 1 : x.find(".")]) for x in os.lis   │
│   340 │   │   │   load_path: Path = load_dir / f"step-{load_step:09d}.ckpt"                      │
│   341 │   │   │   assert load_path.exists(), f"Checkpoint {load_path} does not exist"            │
│   342 │   │   │   loaded_state = torch.load(load_path, map_location="cpu")                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: invalid literal for int() with base 10: 'config'

And here is the config file it seems to be erroring out on:

!!python/object:in2n.in2n_trainer.InstructNeRF2NeRFTrainerConfig
_target: !!python/name:in2n.in2n_trainer.InstructNeRF2NeRFTrainer ''
data: &id002 !!python/object/apply:pathlib.PosixPath
- outputs
- office
experiment_name: office
load_config: null
load_dir: !!python/object/apply:pathlib.PosixPath
- outputs
- office
- nerfacto
- 2023-04-12_102834
load_step: null
log_gradients: false
logging: !!python/object:nerfstudio.configs.base_config.LoggingConfig
  enable_profiler: true
  local_writer: !!python/object:nerfstudio.configs.base_config.LocalWriterConfig
    _target: !!python/name:nerfstudio.utils.writer.LocalWriter ''
    enable: true
    max_log_size: 10
    stats_to_track: !!python/tuple
    - !!python/object/apply:nerfstudio.utils.writer.EventName
      - Train Iter (time)
    - !!python/object/apply:nerfstudio.utils.writer.EventName
      - Train Rays / Sec
    - !!python/object/apply:nerfstudio.utils.writer.EventName
      - Test PSNR
    - !!python/object/apply:nerfstudio.utils.writer.EventName
      - Vis Rays / Sec
    - !!python/object/apply:nerfstudio.utils.writer.EventName
      - Test Rays / Sec
    - !!python/object/apply:nerfstudio.utils.writer.EventName
      - ETA (time)
  max_buffer_size: 20
  relative_log_dir: !!python/object/apply:pathlib.PosixPath []
  steps_per_log: 10
machine: !!python/object:nerfstudio.configs.base_config.MachineConfig
  dist_url: auto
  machine_rank: 0
  num_gpus: 1
  num_machines: 1
  seed: 42
max_num_iterations: 30000
method_name: in2n
mixed_precision: true
optimizers:
  fields:
    optimizer: !!python/object:nerfstudio.engine.optimizers.AdamOptimizerConfig
      _target: &id001 !!python/name:torch.optim.adam.Adam ''
      eps: 1.0e-15
      lr: 0.01
      max_norm: null
      weight_decay: 0
    scheduler: null
  proposal_networks:
    optimizer: !!python/object:nerfstudio.engine.optimizers.AdamOptimizerConfig
      _target: *id001
      eps: 1.0e-15
      lr: 0.01
      max_norm: null
      weight_decay: 0
    scheduler: null
output_dir: !!python/object/apply:pathlib.PosixPath
- outputs
pipeline: !!python/object:in2n.in2n_pipeline.InstructNeRF2NeRFPipelineConfig
  _target: !!python/name:in2n.in2n_pipeline.InstructNeRF2NeRFPipeline ''
  datamanager: !!python/object:in2n.in2n_datamanager.InstructNeRF2NeRFDataManagerConfig
    _target: !!python/name:in2n.in2n_datamanager.InstructNeRF2NeRFDataManager ''
    camera_optimizer: !!python/object:nerfstudio.cameras.camera_optimizers.CameraOptimizerConfig
      _target: !!python/name:nerfstudio.cameras.camera_optimizers.CameraOptimizer ''
      mode: SO3xR3
      optimizer: !!python/object:nerfstudio.engine.optimizers.AdamOptimizerConfig
        _target: *id001
        eps: 1.0e-08
        lr: 1.0e-30
        max_norm: null
        weight_decay: 0.01
      orientation_noise_std: 0.0
      param_group: camera_opt
      position_noise_std: 0.0
      scheduler: !!python/object:nerfstudio.engine.schedulers.ExponentialDecaySchedulerConfig
        _target: !!python/name:nerfstudio.engine.schedulers.ExponentialDecayScheduler ''
        lr_final: null
        lr_pre_warmup: 1.0e-08
        max_steps: 10000
        ramp: cosine
        warmup_steps: 0
    camera_res_scale_factor: 1.0
    data: *id002
    dataparser: !!python/object:nerfstudio.data.dataparsers.nerfstudio_dataparser.NerfstudioDataParserConfig
      _target: !!python/name:nerfstudio.data.dataparsers.nerfstudio_dataparser.Nerfstudio ''
      auto_scale_poses: true
      center_method: poses
      data: !!python/object/apply:pathlib.PosixPath
      - data
      - nerfstudio
      - poster
      depth_unit_scale_factor: 0.001
      downscale_factor: null
      orientation_method: up
      scale_factor: 1.0
      scene_scale: 1.0
      train_split_fraction: 0.9
    eval_image_indices: !!python/tuple
    - 0
    eval_num_images_to_sample_from: -1
    eval_num_rays_per_batch: 4096
    eval_num_times_to_repeat_images: -1
    patch_size: 32
    train_num_images_to_sample_from: -1
    train_num_rays_per_batch: 16384
    train_num_times_to_repeat_images: -1
  diffusion_steps: 20
  edit_count: 1
  edit_rate: 10
  guidance_scale: 12.0
  image_guidance_scale: 1.5
  ip2p_device: null
  ip2p_use_full_precision: true
  lower_bound: 0.02
  model: !!python/object:in2n.in2n.InstructNeRF2NeRFModelConfig
    _target: !!python/name:in2n.in2n.InstructNeRF2NeRFModel ''
    background_color: last_sample
    collider_params:
      far_plane: 6.0
      near_plane: 2.0
    disable_scene_contraction: false
    distortion_loss_mult: 0.002
    enable_collider: true
    eval_num_rays_per_chunk: 32768
    far_plane: 1000.0
    hidden_dim: 64
    hidden_dim_color: 64
    hidden_dim_transient: 64
    interlevel_loss_mult: 1.0
    log2_hashmap_size: 19
    loss_coefficients:
      rgb_loss_coarse: 1.0
      rgb_loss_fine: 1.0
    lpips_loss_mult: 1.0
    max_res: 2048
    near_plane: 0.05
    num_levels: 16
    num_nerf_samples_per_ray: 48
    num_proposal_iterations: 2
    num_proposal_samples_per_ray: !!python/tuple
    - 256
    - 96
    orientation_loss_mult: 0.0001
    patch_size: 32
    pred_normal_loss_mult: 0.001
    predict_normals: false
    proposal_initial_sampler: piecewise
    proposal_net_args_list:
    - hidden_dim: 16
      log2_hashmap_size: 17
      max_res: 128
      num_levels: 5
      use_linear: false
    - hidden_dim: 16
      log2_hashmap_size: 17
      max_res: 256
      num_levels: 5
      use_linear: false
    proposal_update_every: 5
    proposal_warmup: 5000
    proposal_weights_anneal_max_num_iters: 1000
    proposal_weights_anneal_slope: 10.0
    use_average_appearance_embedding: true
    use_l1: true
    use_lpips: true
    use_proposal_weight_anneal: true
    use_same_proposal_network: false
    use_single_jitter: true
  prompt: make the floor lava
  upper_bound: 0.98
relative_model_dir: !!python/object/apply:pathlib.PosixPath
- nerfstudio_models
save_only_latest_checkpoint: true
steps_per_eval_all_images: 25000
steps_per_eval_batch: 1000
steps_per_eval_image: 100
steps_per_save: 250
timestamp: 2023-04-12_110925
viewer: !!python/object:nerfstudio.configs.base_config.ViewerConfig
  ip_address: 127.0.0.1
  launch_bridge_server: true
  max_num_display_images: 512
  num_rays_per_chunk: 32768
  quit_on_train_completion: false
  relative_log_filename: viewer_log_filename.txt
  start_train: true
  websocket_port: 7007
  zmq_port: null
vis: viewer
ayaanzhaque commented 1 year ago

Hi, your error is the file you are importing. It should be the nerfstudio_models folder.

Try this:

ns-train in2n --data outputs/office --load-dir outputs/office/nerfacto/2023-04-12_102834/nerfstudio_models/ --pipeline.prompt 'make the floor lava' --pipeline.guidance-scale 7.5 --pipeline.image-guidance-scale 1.5

samsullivandelgobbo commented 1 year ago

Thanks for the help. Sorry for the oversight