run-demo.sh Error

[MilesTheProwler]

Hi, can someone help me? I have errors when I run bash run-demo.sh.

My python version is 3.8.16, Torch : 2.0.0+cu118 Ubuntu : 22.04

Traceback (most recent call last):
  File "train.py", line 49, in <module>
    main()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/main.py", line 48, in decorated_main
    _run_hydra(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 377, in _run_hydra
    run_and_report(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 88, in run
    cfg = self.compose_config(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 559, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration
    return self._load_configuration_impl(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 239, in _load_configuration_impl
    defaults_list = create_defaults_list(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 748, in create_defaults_list
    defaults, tree = _create_defaults_list(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 718, in _create_defaults_list
    defaults_tree = _create_defaults_tree(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 356, in _create_defaults_tree
    ret = _create_defaults_tree_impl(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 455, in _create_defaults_tree_impl
    return _expand_virtual_root(repo, root, overrides, skip_missing)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 280, in _expand_virtual_root
    subtree = _create_defaults_tree_impl(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 575, in _create_defaults_tree_impl
    add_child(children, new_root)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 518, in add_child
    subtree_ = _create_defaults_tree_impl(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 486, in _create_defaults_tree_impl
    config_not_found_error(repo=repo, tree=root)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 802, in config_not_found_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: In 'SNARF_NGP': Could not find 'dataset/'

Config search path:
        provider=hydra, path=pkg://hydra.conf
        provider=main, path=file:///mnt/c/Users/ADMIN/Desktop/InstantAvatar/confs
        provider=schema, path=structured://
Global seed set to 42
Switch to /mnt/c/Users/ADMIN/Desktop/InstantAvatar/outputs/peoplesnapshot/demo/male-3-casual
[train] Loading from /mnt/c/Users/ADMIN/Desktop/InstantAvatar/data/PeopleSnapshot/male-3-casual/poses/anim_nerf_train.npz
[val] Loading from /mnt/c/Users/ADMIN/Desktop/InstantAvatar/data/PeopleSnapshot/male-3-casual/poses/anim_nerf_val.npz
[test] Loading from /mnt/c/Users/ADMIN/Desktop/InstantAvatar/data/PeopleSnapshot/male-3-casual/poses/anim_nerf_test.npz
[2023-08-06 21:31:30,205][torch.distributed.nn.jit.instantiator][INFO] - Created a temporary directory at /tmp/tmp7ujfp0hu
[2023-08-06 21:31:30,205][torch.distributed.nn.jit.instantiator][INFO] - Writing /tmp/tmp7ujfp0hu/_remote_module_non_scriptable.py
Error executing job with overrides: ['dataset=peoplesnapshot/male-3-casual', 'experiment=demo']
Traceback (most recent call last):
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 583, in _locate
    obj = getattr(obj, part)
AttributeError: module 'instant_avatar.renderers' has no attribute 'raymarcher_acc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 589, in _locate
    obj = import_module(mod)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/mnt/c/Users/ADMIN/Desktop/InstantAvatar/instant_avatar/renderers/raymarcher_acc.py", line 1, in <module>
    from ..models.structures.density_grid import DensityGrid
  File "/mnt/c/Users/ADMIN/Desktop/InstantAvatar/instant_avatar/models/structures/density_grid.py", line 4, in <module>
    import kaolin
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/kaolin/__init__.py", line 1, in <module>
    raise ImportError("This is the kaolin placeholder wheel from https://pypi.org/project/kaolin/. Please follow https://kaolin.readthedocs.io/en/latest/notes/installation.html for installation.")
ImportError: This is the kaolin placeholder wheel from https://pypi.org/project/kaolin/. Please follow https://kaolin.readthedocs.io/en/latest/notes/installation.html for installation.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 130, in _resolve_target
    target = _locate(target)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 597, in _locate
    raise ImportError(
ImportError: Error loading 'instant_avatar.renderers.raymarcher_acc.Raymarcher':
ImportError('This is the kaolin placeholder wheel from https://pypi.org/project/kaolin/. Please follow https://kaolin.readthedocs.io/en/latest/notes/installation.html for installation.')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 86, in _call_target
    return _target_(*args, **kwargs)
  File "/mnt/c/Users/ADMIN/Desktop/InstantAvatar/instant_avatar/models/DNeRF.py", line 27, in __init__
    self.renderer = hydra.utils.instantiate(opt.renderer, smpl_init=opt.get("smpl_init", False))
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 218, in instantiate
    return instantiate_node(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 321, in instantiate_node
    _target_ = _resolve_target(node.get(_Keys.TARGET), full_key)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 135, in _resolve_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error locating target 'instant_avatar.renderers.raymarcher_acc.Raymarcher', see chained exception above.
full_key: opt.renderer

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "animate.py", line 121, in <module>
    main()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/main.py", line 48, in decorated_main
    _run_hydra(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 377, in _run_hydra
    run_and_report(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 111, in run
    _ = ret.return_value
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "animate.py", line 88, in main
    model = hydra.utils.instantiate(opt.model, datamodule=datamodule, _recursive_=False)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 218, in instantiate
    return instantiate_node(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 331, in instantiate_node
    return _call_target(_target_, partial, args, kwargs, full_key)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 91, in _call_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'instant_avatar.models.DNeRF.DNeRFModel':
InstantiationException("Error locating target 'instant_avatar.renderers.raymarcher_acc.Raymarcher', see chained exception above.\nfull_key: opt.renderer")
full_key: model

[tijiang13]

Hi,

According to the error message:

raise ImportError("This is the kaolin placeholder wheel from https://pypi.org/project/kaolin/. Please follow https://kaolin.readthedocs.io/en/latest/notes/installation.html for installation.")
ImportError: This is the kaolin placeholder wheel from https://pypi.org/project/kaolin/. Please follow https://kaolin.readthedocs.io/en/latest/notes/installation.html for installation.

It appears that the installation of kaolin is problematic.

Best, Tianjian

[MilesTheProwler]

@tijiang13 thank you for your reply. Can I get your social account? I have something to ask about my custom data. Could you please help me. Currently, I am doing my master degree research on this.

Can I also use mask rvm instead of using sam ?

HYDRA_FULL_ERROR=1 python fit.py

Global seed set to 42
Switch to /mnt/c/Users/ADMIN/Desktop/InstantAvatar/bash/outputs/neuman/test
[train] No optimized smpl found.
[val] No optimized smpl found.
[test] No optimized smpl found.
[2023-08-08 00:09:26,193][torch.distributed.nn.jit.instantiator][INFO] - Created a temporary directory at /tmp/tmpxjkg3mtq
[2023-08-08 00:09:26,194][torch.distributed.nn.jit.instantiator][INFO] - Writing /tmp/tmpxjkg3mtq/_remote_module_non_scriptable.py
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
Loading model from: /mnt/c/Users/ADMIN/Desktop/InstantAvatar/bash/third_parties/lpips/weights/v0.1/vgg.pth
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
Saving configs.
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

  | Name       | Type               | Params
--------------------------------------------------
0 | net_coarse | NeRFNGPNet         | 13.0 M
1 | SMPL_param | SMPLParamEmbedding | 1.5 K
2 | loss_fn    | NGPLoss            | 14.7 M
3 | renderer   | Raymarcher         | 0
--------------------------------------------------
13.0 M    Trainable params
14.7 M    Non-trainable params
27.8 M    Total params
111.016   Total estimated model params size (MB)
Error executing job with overrides: ['dataset=neuman/test', 'experiment=neuman/test/exp', 'deformer=smpl', 'train.max_epochs=200']
Traceback (most recent call last):
  File "fit.py", line 78, in <module>
    main()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/main.py", line 48, in decorated_main
    _run_hydra(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 377, in _run_hydra
    run_and_report(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 111, in run
    _ = ret.return_value
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "fit.py", line 50, in main
    trainer.fit(model)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 738, in fit
    self._call_and_handle_interrupt(
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 683, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 773, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1195, in _run
    self._dispatch()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1275, in _dispatch
    self.training_type_plugin.start_training(self)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
    self._results = trainer.run_stage()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1285, in run_stage
    return self._run_train()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1315, in _run_train
    self.fit_loop.run()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 140, in run
    self.on_run_start(*args, **kwargs)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 197, in on_run_start
    self.trainer.reset_train_val_dataloaders(self.trainer.lightning_module)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/data_loading.py", line 569, in reset_train_val_dataloaders
    self.reset_train_dataloader(model=model)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/data_loading.py", line 345, in reset_train_dataloader
    self.train_dataloader = self.request_dataloader(RunningStage.TRAINING, model=model)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/data_loading.py", line 585, in request_dataloader
    dataloader = source.dataloader()
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 296, in dataloader
    return self.instance.trainer.call_hook(self.name, pl_module=self.instance)
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1497, in call_hook
    output = model_fx(*args, **kwargs)
  File "/mnt/c/Users/ADMIN/Desktop/InstantAvatar/bash/instant_avatar/models/DNeRF.py", line 246, in train_dataloader
    return self.datamodule.train_dataloader()
  File "/mnt/c/Users/ADMIN/Desktop/InstantAvatar/bash/instant_avatar/datasets/custom.py", line 164, in train_dataloader
    return DataLoader(self.trainset,
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 351, in __init__
    sampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]
  File "/home/thar/miniconda3/envs/TestAvatar/lib/python3.8/site-packages/torch/utils/data/sampler.py", line 107, in __init__
    raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0

Thanks.

[tijiang13]

Hi,

Unfortunately I barely use any social media. However, feel free to leave your questions here, and I'll try to get back to you.

Regarding the error message, it looks like the dataloader is unable to locate any data at the specified location. Please ensure that the required data is placed correctly and that the file paths are accurate.

Best, Tianjian

[MilesTheProwler]

Thank you for your reply. Can I know where should I put my custom dataset in order to match with your repo dataset location ?

In addition, I saw that your repo can output 3D obj by using marching cube, do you have any obj example from marching cube? If so could you please share with me , I want to see the result of obj file.

Could you also please check about my custom dataset is correct or not ?

[tijiang13]

You can run the demo code and refer https://github.com/tijiang13/InstantAvatar/issues/39 for details.

For the image you show, it appears that you didn't run the SAM.

Best, Tianjian

[MilesTheProwler]

I cannot run SAM got erorr, so I run RVM, Is it same right ?

[tijiang13]

No it's not.

Btw if you are looking for the mesh this is probably the wrong repo. Generally the mesh produced by NeRF is very noisy.

Best, Tianjian

[MilesTheProwler]

Could you give me some obj file that is obtained from NeRF and convert with marching cube ? I want to see the result of obj file.

Thanks.

[tijiang13]

I don't have that at hand. But you can try the snippet in quick-start and the script in #39 which do not require the installation of SAM and should get you some results in 10 minutes.

Best, Tianjian

[MilesTheProwler]

Thank you !