search

nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
9.45k stars 1.29k forks source link

Error integrating custom data parser #3491

Open lyg1597 opened 4 days ago

lyg1597 commented 4 days ago

Describe the bug I am currently trying to implement my own variant of splafacto which include a custom data parser. However, while I am trying to run ns-install-cli I got an error saying subprocess.CalledProcessError: Command '['ns-train', '--tyro-print-completion', 'zsh']' returned non-zero exit status 1.. This also breaks ns-train -h as I am getting error AssertionError:pipeline.datamanager.dataparserwas provided a default value of type <class 'splatfactoenv.splatfactoenv_dataparser.SplatfactoEnvDataParserConfig'> but no matching subcommand was found.. I am running the code on wsl 20.04 with python 3.8 in conda environment. The nerfstudio version is 1.1.4.

I am sort of confident about my code as the exact same code works on a ubuntu 20.04 machine with python 3.8, nerfstudio version 1.1.3 inside a virtual environment, but I am getting warning /home/younger/work/nerfstudio/venv/lib/python3.8/site-packages/tyro/_resolver.py:437: UserWarning: <class 'splatfactoenv.splatfactoenv_dataparser.SplatfactoEnvDataParserConfig'> does not match any type in Union: which looks similar to the error I am getting.

To Reproduce Steps to reproduce the behavior:

  1. Clone my code from https://github.com/lyg1597/splatfacto-env.git
  2. In the folder run python3 -m pip install -e .
  3. In the same folder, run ns-install-cli.

Additional context I think the problem is coming from splatfactoenv_config.py:L38 and splatfactoenv_datamanager.py:L29 as I require the datapaser field SplatfactoEnvDataManagerConfig to have type AnnotatedDataParserUnion which doesn't include the data parser I am using (SplatfactoEnvDataParserConfig). However, I am wondering how I should solve this issue?

Full error message

(nerfstudio) younger@DESKTOP-7IAS1JU:/mnt/d/1_study/1_research/nerfstudio$ ns-train -h
/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/field_components/activations.py:32: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @custom_fwd(cast_inputs=torch.float32)
/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/field_components/activations.py:39: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, g):
Traceback (most recent call last):
  File "/home/younger/anaconda3/envs/nerfstudio/bin/ns-train", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 263, in entrypoint
    tyro.cli(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_cli.py", line 209, in cli
    output = _cli_impl(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_cli.py", line 415, in _cli_impl
    parser_spec = _parsers.ParserSpecification.from_callable_or_type(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 117, in from_callable_or_type
    field_out = handle_field(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 311, in handle_field
    subparsers_attempt = SubparsersSpecification.from_field(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 524, in from_field
    subparser = ParserSpecification.from_callable_or_type(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 117, in from_callable_or_type
    field_out = handle_field(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 336, in handle_field
    return ParserSpecification.from_callable_or_type(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 117, in from_callable_or_type
    field_out = handle_field(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 336, in handle_field
    return ParserSpecification.from_callable_or_type(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 117, in from_callable_or_type
    field_out = handle_field(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 311, in handle_field
    subparsers_attempt = SubparsersSpecification.from_field(
  File "/home/younger/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/tyro/_parsers.py", line 468, in from_field
    assert default_name is not None, (
AssertionError: `pipeline.datamanager.dataparser` was provided a default value of type <class 'splatfactoenv.splatfactoenv_dataparser.SplatfactoEnvDataParserConfig'> but no matching subcommand was found. A type may be missing in the Union type declaration for `pipeline.datamanager.dataparser`, which currently expects [typing_extensions.Annotated[nerfstudio.data.dataparsers.nerfstudio_dataparser.NerfstudioDataParserConfig, _SubcommandConfiguration(name='nerfstudio-data', default=NerfstudioDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.nerfstudio_dataparser.Nerfstudio'>, data=PosixPath('.'), scale_factor=1.0, downscale_factor=None, scene_scale=1.0, orientation_method='up', center_method='poses', auto_scale_poses=True, eval_mode='fraction', train_split_fraction=0.9, eval_interval=8, depth_unit_scale_factor=0.001, mask_color=None, load_3D_points=False), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.minimal_dataparser.MinimalDataParserConfig, _SubcommandConfiguration(name='minimal-parser', default=MinimalDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.minimal_dataparser.MinimalDataParser'>, data=PosixPath('/home/nikhil/nerfstudio-main/tests/data/lego_test/minimal_parser')), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.arkitscenes_dataparser.ARKitScenesDataParserConfig, _SubcommandConfiguration(name='arkit-data', default=ARKitScenesDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.arkitscenes_dataparser.ARKitScenes'>, data=PosixPath('data/ARKitScenes/3dod/Validation/41069021'), scale_factor=1.0, scene_scale=1.0, center_method='poses', auto_scale_poses=True, train_split_fraction=0.9, depth_unit_scale_factor=0.001), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.blender_dataparser.BlenderDataParserConfig, _SubcommandConfiguration(name='blender-data', default=BlenderDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.blender_dataparser.Blender'>, data=PosixPath('data/blender/lego'), scale_factor=1.0, alpha_color='white', ply_path=None), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.instant_ngp_dataparser.InstantNGPDataParserConfig, _SubcommandConfiguration(name='instant-ngp-data', default=InstantNGPDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.instant_ngp_dataparser.InstantNGP'>, data=PosixPath('data/ours/posterv2'), scene_scale=0.3333, eval_mode='fraction', train_split_fraction=0.9, eval_interval=8), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.nuscenes_dataparser.NuScenesDataParserConfig, _SubcommandConfiguration(name='nuscenes-data', default=NuScenesDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.nuscenes_dataparser.NuScenes'>, data=PosixPath('scene-0103'), data_dir=PosixPath('/mnt/local/NuScenes'), version='v1.0-mini', cameras=('FRONT',), mask_dir=None, train_split_fraction=0.9, verbose=False), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.dnerf_dataparser.DNeRFDataParserConfig, _SubcommandConfiguration(name='dnerf-data', default=DNeRFDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.dnerf_dataparser.DNeRF'>, data=PosixPath('data/dnerf/lego'), scale_factor=1.0, alpha_color='white'), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.phototourism_dataparser.PhototourismDataParserConfig, _SubcommandConfiguration(name='phototourism-data', default=PhototourismDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.phototourism_dataparser.Phototourism'>, data=PosixPath('data/phototourism/brandenburg-gate'), scale_factor=3.0, alpha_color='white', train_split_fraction=0.9, scene_scale=1.0, orientation_method='up', center_method='poses', auto_scale_poses=True), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.dycheck_dataparser.DycheckDataParserConfig, _SubcommandConfiguration(name='dycheck-data', default=DycheckDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.dycheck_dataparser.Dycheck'>, data=PosixPath('data/iphone/mochi-high-five'), scale_factor=5.0, alpha_color='white', downscale_factor=1, scene_box_bound=1.5), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.scannet_dataparser.ScanNetDataParserConfig, _SubcommandConfiguration(name='scannet-data', default=ScanNetDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.scannet_dataparser.ScanNet'>, data=PosixPath('nvsmask3d/data/scene_example'), scale_factor=1.0, scene_scale=1.0, center_method='poses', auto_scale_poses=True, train_split_fraction=0.9, depth_unit_scale_factor=0.001, load_3D_points=True, point_cloud_color=True, ply_file_path=PosixPath('nvsmask3d/data/scene_example/scene_example.ply')), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.sdfstudio_dataparser.SDFStudioDataParserConfig, _SubcommandConfiguration(name='sdfstudio-data', default=SDFStudioDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.sdfstudio_dataparser.SDFStudio'>, data=PosixPath('data/DTU/scan65'), include_mono_prior=False, depth_unit_scale_factor=0.001, include_foreground_mask=False, downscale_factor=1, scene_scale=2.0, skip_every_for_val_split=1, auto_orient=True), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.nerfosr_dataparser.NeRFOSRDataParserConfig, _SubcommandConfiguration(name='nerfosr-data', default=NeRFOSRDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.nerfosr_dataparser.NeRFOSR'>, data=PosixPath('data/NeRF-OSR/Data'), scene='stjacob', scene_scale=1.0, scale_factor=1.0, use_masks=False, orientation_method='vertical', center_method='focus', auto_scale_poses=True), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.sitcoms3d_dataparser.Sitcoms3DDataParserConfig, _SubcommandConfiguration(name='sitcoms3d-data', default=Sitcoms3DDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.sitcoms3d_dataparser.Sitcoms3D'>, data=PosixPath('data/sitcoms3d/TBBT-big_living_room'), include_semantics=True, downscale_factor=4, scene_scale=2.0), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.scannetpp_dataparser.ScanNetppDataParserConfig, _SubcommandConfiguration(name='scannetpp-data', default=ScanNetppDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.scannetpp_dataparser.ScanNetpp'>, data=PosixPath('scannetpp/410c470782'), scale_factor=1.0, scene_scale=1.5, orientation_method='up', center_method='poses', auto_scale_poses=True, images_dir=PosixPath('dslr/resized_images'), masks_dir=PosixPath('dslr/resized_anon_masks'), transforms_path=PosixPath('dslr/nerfstudio/transforms.json')), description='', prefix_name=False, constructor_factory=None)], typing_extensions.Annotated[nerfstudio.data.dataparsers.colmap_dataparser.ColmapDataParserConfig, _SubcommandConfiguration(name='colmap', default=ColmapDataParserConfig(_target=<class 'nerfstudio.data.dataparsers.colmap_dataparser.ColmapDataParser'>, data=PosixPath('.'), scale_factor=1.0, downscale_factor=None, downscale_rounding_mode='floor', scene_scale=1.0, orientation_method='up', center_method='poses', auto_scale_poses=True, assume_colmap_world_coordinate_convention=True, eval_mode='interval', train_split_fraction=0.9, eval_interval=8, depth_unit_scale_factor=0.001, images_path=PosixPath('images'), masks_path=None, depths_path=None, colmap_path=PosixPath('colmap/sparse/0'), load_3D_points=True, max_2D_matches_per_3D_point=0), description='', prefix_name=False, constructor_factory=None)]].
lyg1597 commented 4 days ago

I managed to solve the problem by adding a custom union type 

CustomDataParserUnion = Union[SplatfactoEnvDataParserConfig, AnnotatedDataParserUnion]

and use it as the type hint for dataparser field in SplatfactoEnvDataManagerConfig.

FavorMylikes commented 4 days ago

@lyg1597 I solved this issue by downgrading tyro version

pip install tyro==0.8.12

I don't know what happened, but I think it relates to the tyro upgrade 3 days ago.

TouKaienn commented 3 days ago

@lyg1597 I solved this issue by downgrading typo version

pip install typo==0.8.12

I don't know what happened, but I think it relates to the typo upgrade 3 days ago.

This solved my problem, too! But note that it is not "typo" package, its name is "tyro". The package name here is a "typo" :laughing: So you should actually install it with:

pip install tyro==0.8.12
FavorMylikes commented 3 days ago

@lyg1597 I solved this issue by downgrading typo version

pip install typo==0.8.12

I don't know what happened, but I think it relates to the typo upgrade 3 days ago.

This solved my problem, too! But note that it is not "typo" package, its name is "tyro". The package name here is a "typo" 😆 So you should actually install it with:

pip install tyro==0.8.12

LoL, look at what a typo I made