tijiang13 / InstantAvatar

InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds (CVPR 2023)
365 stars 31 forks source link

Error in fit.py #31

Closed Net-Maker closed 1 year ago

Net-Maker commented 1 year ago

Thanks for your excellent work! When I was running neuman data,I met this error:

Traceback (most recent call last):
  File "fit.py", line 78, in <module>
    main()
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/main.py", line 48, in decorated_main
    _run_hydra(
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 377, in _run_hydra
    run_and_report(
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 294, in run_and_report
    raise ex
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 111, in run
    _ = ret.return_value
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "fit.py", line 50, in main
    trainer.fit(model)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 738, in fit
    self._call_and_handle_interrupt(
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 683, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 773, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1195, in _run
    self._dispatch()
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1275, in _dispatch
    self.training_type_plugin.start_training(self)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
    self._results = trainer.run_stage()
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1285, in run_stage
    return self._run_train()
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1315, in _run_train
    self.fit_loop.run()
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 145, in run
    self.advance(*args, **kwargs)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 234, in advance
    self.epoch_loop.run(data_fetcher)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 145, in run
    self.advance(*args, **kwargs)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 193, in advance
    batch_output = self.batch_loop.run(batch, batch_idx)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 145, in run
    self.advance(*args, **kwargs)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 90, in advance
    outputs = self.manual_loop.run(split_batch, batch_idx)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 145, in run
    self.advance(*args, **kwargs)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/manual_loop.py", line 111, in advance
    training_step_output = self.trainer.accelerator.training_step(step_kwargs)
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 216, in training_step
    return self.training_type_plugin.training_step(*step_kwargs.values())
  File "/home/wjx/anaconda3/envs/avatar/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 213, in training_step
    return self.model.training_step(*args, **kwargs)
  File "/home/wjx/Desktop/money/InstantAvatar/instant_avatar/models/DNeRF.py", line 129, in training_step
    reg = self.update_density_grid()
  File "/home/wjx/Desktop/money/InstantAvatar/instant_avatar/models/DNeRF.py", line 100, in update_density_grid
    _, density, valid = self.renderer.density_grid_train.update(self.deformer,
  File "/home/wjx/Desktop/money/InstantAvatar/instant_avatar/models/structures/density_grid.py", line 39, in update
    _, density = deformer(coords, net, eval_mode=False)
  File "/home/wjx/Desktop/money/InstantAvatar/instant_avatar/deformers/smpl_deformer.py", line 137, in __call__
    return self.deform_train(pts, model)
  File "/home/wjx/Desktop/money/InstantAvatar/instant_avatar/deformers/smpl_deformer.py", line 113, in deform_train
    pts_cano, valid = self.deform(pts)
  File "/home/wjx/Desktop/money/InstantAvatar/instant_avatar/deformers/smpl_deformer.py", line 95, in deform
    dist_sq, idx = ops.knn_points(pts.float(), self.vertices.float(), K=self.k)
ValueError: too many values to unpack (expected 2)

I am using the 'seattle in' Neuman data,and I want to use just 41 pictures to test the performance. The process-sequence.sh is running smoothly. May the number of data cause this error? And I found that you write this function"knn_points" by yourself with CUDA,if you can recollect the implement details, could you please tell me why does this happen? Looking forward to your reply.

tijiang13 commented 1 year ago

Hi Net-Maker,

I have ported the knn function from PyTorch3D as some reported it inconvenient to install PyTorch3D. However, I forgot to update the API accordingly. This should be resolved as of commit c79743dafb9297a4614a3fd10ca382ccac997fd9 (thanks to @Miles629).

Best, Tianjian