CLI parser doesn't correctly parse namespaced args

tayden commented 2 hours ago

I'm getting and error: "ValueError: not enough values to unpack (expected 4, got 3)" from the CLI argument parser when passing namespaced args like --data.parser.split_mode "reconstruction" in the CLI.

This occurs when using, e.g. downscaled images with --data.parser.down_sample_factor 4 or running the recommended SpotLessSplats command from the docs with --data.parser.split_mode "reconstruction". I assume this happens with other namespaced arguments as well.

The entire Traceback is:

$ python main.py fit --viewer --config configs/spot_less_splats/gsplat-cluster.yaml --data.path /home/taylor/office/dense --data.parser Colmap --data.parser.split_mode="reconstruction" -n office_sls

Traceback (most recent call last):
  File "/home/taylor/gaussian-splatting-lightning/main.py", line 4, in <module>
    cli()
  File "/home/taylor/gaussian-splatting-lightning/internal/entrypoints/gspl.py", line 12, in cli
    CLI(
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 349, in __init__
    self.parse_arguments(self.parser, args)
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 499, in parse_arguments
    self.config = parser.parse_args(args)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_deprecated.py", line 123, in patched_parse
    cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_core.py", line 386, in parse_args
    cfg, unk = self.parse_known_args(args=args, namespace=cfg)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_core.py", line 257, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/argparse.py", line 2110, in _parse_known_args
    positionals_end_index = consume_positionals(start_index)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/argparse.py", line 2087, in consume_positionals
    take_action(action, args)
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/argparse.py", line 1983, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_actions.py", line 643, in __call__
    namespace[subcommand] = subparser.parse_args(arg_strings, namespace=subnamespace, **kwargs)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_deprecated.py", line 123, in patched_parse
    cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_core.py", line 386, in parse_args
    cfg, unk = self.parse_known_args(args=args, namespace=cfg)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/site-packages/jsonargparse/_core.py", line 257, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/argparse.py", line 2128, in _parse_known_args
    start_index = consume_optional(start_index)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniforge3/envs/gs2/lib/python3.11/argparse.py", line 1990, in consume_optional
    action, option_string, sep, explicit_arg = option_tuple
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 4, got 3)

I'm using Python 3.11 and am up to date at main branch commit id: 13c8d2923cef4d7755311f2dff4ed489910932d1

The issue occurs in both Bash and Fish shell on Ubuntu 22.04.

My conda environment:

``` # packages in environment at /home/taylor/miniforge3/envs/gs2: # # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu absl-py 2.1.0 pypi_0 pypi accelerate 0.34.2 pypi_0 pypi aiohappyeyeballs 2.4.0 pypi_0 pypi aiohttp 3.10.5 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi annotated-types 0.7.0 pypi_0 pypi antlr4-python3-runtime 4.9.3 pypi_0 pypi anyio 4.5.0 pypi_0 pypi arrow 1.3.0 pypi_0 pypi asttokens 2.4.1 pypi_0 pypi attrs 24.2.0 pypi_0 pypi backoff 2.2.1 pypi_0 pypi beautifulsoup4 4.12.3 pypi_0 pypi blas 1.0 mkl blessed 1.20.0 pypi_0 pypi boto3 1.35.23 pypi_0 pypi botocore 1.35.23 pypi_0 pypi brotli-python 1.0.9 py311h6a678d5_8 bzip2 1.0.8 h5eee18b_6 ca-certificates 2024.7.2 h06a4308_0 certifi 2024.8.30 py311h06a4308_0 chardet 5.2.0 pypi_0 pypi charset-normalizer 3.3.2 pyhd3eb1b0_0 click 8.1.7 pypi_0 pypi colorlog 6.8.2 pypi_0 pypi contourpy 1.3.0 pypi_0 pypi croniter 1.4.1 pypi_0 pypi cuda-cudart 12.1.105 0 nvidia cuda-cupti 12.1.105 0 nvidia cuda-libraries 12.1.0 0 nvidia cuda-nvrtc 12.1.105 0 nvidia cuda-nvtx 12.1.105 0 nvidia cuda-opencl 12.6.68 0 nvidia cuda-runtime 12.1.0 0 nvidia cuda-version 12.6 3 nvidia cycler 0.12.1 pypi_0 pypi dateutils 0.6.12 pypi_0 pypi decorator 5.1.1 pypi_0 pypi deepdiff 7.0.1 pypi_0 pypi diff-gaussian-rasterization 0.0.0 pypi_0 pypi diffusers 0.27.2 pypi_0 pypi docker-pycreds 0.4.0 pypi_0 pypi docstring-parser 0.16 pypi_0 pypi editor 1.6.6 pypi_0 pypi embreex 2.17.7.post5 pypi_0 pypi executing 2.1.0 pypi_0 pypi fastapi 0.115.0 pypi_0 pypi ffmpeg 4.3 hf484d3e_0 pytorch filelock 3.13.1 py311h06a4308_0 fonttools 4.53.1 pypi_0 pypi freetype 2.12.1 h4a9f257_0 frozenlist 1.4.1 pypi_0 pypi fsspec 2024.9.0 pypi_0 pypi gaussian-splatting-lightning 0.0.0 pypi_0 pypi gdown 5.2.0 pypi_0 pypi gitdb 4.0.11 pypi_0 pypi gitpython 3.1.43 pypi_0 pypi gmp 6.2.1 h295c915_3 gmpy2 2.1.2 py311hc9b5ff0_0 gnutls 3.6.15 he1e5248_0 grpcio 1.66.1 pypi_0 pypi gsplat 0.1.13 pypi_0 pypi h11 0.14.0 pypi_0 pypi httpcore 1.0.5 pypi_0 pypi httpx 0.27.2 pypi_0 pypi huggingface-hub 0.25.0 pypi_0 pypi hydra-core 1.3.2 pypi_0 pypi idna 3.7 py311h06a4308_0 imageio 2.35.1 pypi_0 pypi importlib-metadata 8.5.0 pypi_0 pypi importlib-resources 6.4.5 pypi_0 pypi inquirer 3.4.0 pypi_0 pypi intel-openmp 2023.1.0 hdb19cb5_46306 ipython 8.27.0 pypi_0 pypi itsdangerous 2.2.0 pypi_0 pypi jaxtyping 0.2.34 pypi_0 pypi jedi 0.19.1 pypi_0 pypi jinja2 3.1.4 py311h06a4308_0 jmespath 1.0.1 pypi_0 pypi joblib 1.4.2 pypi_0 pypi jpeg 9e h5eee18b_3 jsonargparse 4.24.1 pypi_0 pypi jsonschema 4.23.0 pypi_0 pypi jsonschema-specifications 2023.12.1 pypi_0 pypi kiwisolver 1.4.7 pypi_0 pypi lame 3.100 h7b6447c_0 lazy-loader 0.4 pypi_0 pypi lcms2 2.12 h3be6417_0 ld_impl_linux-64 2.38 h1181459_1 lerc 3.0 h295c915_0 libcublas 12.1.0.26 0 nvidia libcufft 11.0.2.4 0 nvidia libcufile 1.11.1.6 0 nvidia libcurand 10.3.7.68 0 nvidia libcusolver 11.4.4.55 0 nvidia libcusparse 12.0.2.55 0 nvidia libdeflate 1.17 h5eee18b_1 libffi 3.4.4 h6a678d5_1 libgcc-ng 11.2.0 h1234567_1 libgomp 11.2.0 h1234567_1 libiconv 1.16 h5eee18b_3 libidn2 2.3.4 h5eee18b_0 libjpeg-turbo 2.0.0 h9bf148f_0 pytorch libnpp 12.0.2.50 0 nvidia libnvjitlink 12.1.105 0 nvidia libnvjpeg 12.1.1.14 0 nvidia libpng 1.6.39 h5eee18b_0 libstdcxx-ng 11.2.0 h1234567_1 libtasn1 4.19.0 h5eee18b_0 libtiff 4.5.1 h6a678d5_0 libunistring 0.9.10 h27cfd23_0 libuuid 1.41.5 h5eee18b_0 libwebp-base 1.3.2 h5eee18b_0 lightning 2.0.9.post0 pypi_0 pypi lightning-cloud 0.5.70 pypi_0 pypi lightning-utilities 0.11.7 pypi_0 pypi llvm-openmp 14.0.6 h9e868ea_0 lxml 5.3.0 pypi_0 pypi lz4-c 1.9.4 h6a678d5_1 manifold3d 2.5.1 pypi_0 pypi mapbox-earcut 1.0.2 pypi_0 pypi markdown 3.7 pypi_0 pypi markdown-it-py 3.0.0 pypi_0 pypi markupsafe 2.1.3 py311h5eee18b_0 matplotlib 3.9.2 pypi_0 pypi matplotlib-inline 0.1.7 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mediapy 1.2.2 pypi_0 pypi mkl 2023.1.0 h213fc3f_46344 mkl-service 2.4.0 py311h5eee18b_1 mkl_fft 1.3.10 py311h5eee18b_0 mkl_random 1.2.7 py311ha02d727_0 mpc 1.1.0 h10f8cd9_1 mpfr 4.0.2 hb69a4c5_1 mpmath 1.3.0 py311h06a4308_0 msgpack 1.1.0 pypi_0 pypi multidict 6.1.0 pypi_0 pypi ncurses 6.4 h6a678d5_0 nettle 3.7.3 hbbd107a_1 networkx 3.2.1 py311h06a4308_0 ninja 1.11.1.1 pypi_0 pypi nodeenv 1.9.1 pypi_0 pypi numpy 2.0.1 py311h08b1b3b_1 numpy-base 2.0.1 py311hf175353_1 omegaconf 2.3.0 pypi_0 pypi opencv-python 4.10.0.84 pypi_0 pypi openh264 2.1.1 h4ff587b_0 openjpeg 2.5.2 he7f1fd0_0 openssl 3.0.15 h5eee18b_0 ordered-set 4.1.0 pypi_0 pypi packaging 24.1 pypi_0 pypi parso 0.8.4 pypi_0 pypi pexpect 4.9.0 pypi_0 pypi pillow 10.4.0 py311h5eee18b_0 pip 24.2 py311h06a4308_0 platformdirs 4.3.6 pypi_0 pypi plyfile 0.8.1 pypi_0 pypi prompt-toolkit 3.0.47 pypi_0 pypi protobuf 5.28.2 pypi_0 pypi psutil 6.0.0 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.3 pypi_0 pypi pycollada 0.8 pypi_0 pypi pydantic 1.10.18 pypi_0 pypi pydantic-core 2.4.0 pypi_0 pypi pygments 2.18.0 pypi_0 pypi pyjwt 2.9.0 pypi_0 pypi pyliblzfse 0.4.1 pypi_0 pypi pyparsing 3.1.4 pypi_0 pypi pysocks 1.7.1 py311h06a4308_0 python 3.11.9 h955ad1f_0 python-dateutil 2.9.0.post0 pypi_0 pypi python-multipart 0.0.9 pypi_0 pypi pytorch 2.4.1 py3.11_cuda12.1_cudnn9.1.0_0 pytorch pytorch-cuda 12.1 ha16c6d3_5 pytorch pytorch-lightning 2.3.3 pypi_0 pypi pytorch-mutex 1.0 cuda pytorch pytz 2024.2 pypi_0 pypi pyyaml 6.0.1 py311h5eee18b_0 readchar 4.2.0 pypi_0 pypi readline 8.2 h5eee18b_0 referencing 0.35.1 pypi_0 pypi regex 2024.9.11 pypi_0 pypi requests 2.32.3 py311h06a4308_0 rich 13.8.1 pypi_0 pypi rpds-py 0.20.0 pypi_0 pypi rtree 1.3.0 pypi_0 pypi runs 1.2.2 pypi_0 pypi s3transfer 0.10.2 pypi_0 pypi safetensors 0.4.5 pypi_0 pypi scikit-image 0.24.0 pypi_0 pypi scikit-learn 1.5.2 pypi_0 pypi scipy 1.14.1 pypi_0 pypi sentry-sdk 2.14.0 pypi_0 pypi setproctitle 1.3.3 pypi_0 pypi setuptools 75.1.0 py311h06a4308_0 shapely 2.0.6 pypi_0 pypi shtab 1.7.1 pypi_0 pypi simple-knn 0.0.0 pypi_0 pypi six 1.16.0 pypi_0 pypi smmap 5.0.1 pypi_0 pypi sniffio 1.3.1 pypi_0 pypi soupsieve 2.6 pypi_0 pypi splines 0.3.0 pypi_0 pypi sqlite 3.45.3 h5eee18b_0 stack-data 0.6.3 pypi_0 pypi starlette 0.38.5 pypi_0 pypi starsessions 1.3.0 pypi_0 pypi svg-path 6.3 pypi_0 pypi sympy 1.13.2 py311h06a4308_0 tbb 2021.8.0 hdb19cb5_0 tensorboard 2.17.1 pypi_0 pypi tensorboard-data-server 0.7.2 pypi_0 pypi tensorboardx 2.6.2.2 pypi_0 pypi threadpoolctl 3.5.0 pypi_0 pypi tifffile 2024.8.30 pypi_0 pypi tk 8.6.14 h39e8969_0 tokenizers 0.19.1 pypi_0 pypi torchaudio 2.4.1 py311_cu121 pytorch torchmetrics 1.4.2 pypi_0 pypi torchtriton 3.0.0 py311 pytorch torchvision 0.19.1 py311_cu121 pytorch tqdm 4.66.5 pypi_0 pypi traitlets 5.14.3 pypi_0 pypi transformers 4.40.1 pypi_0 pypi trimesh 4.4.9 pypi_0 pypi typeguard 2.13.3 pypi_0 pypi types-python-dateutil 2.9.0.20240906 pypi_0 pypi typeshed-client 2.7.0 pypi_0 pypi typing_extensions 4.11.0 py311h06a4308_0 tyro 0.8.11 pypi_0 pypi tzdata 2024a h04d1e81_0 urllib3 2.2.2 py311h06a4308_0 uvicorn 0.30.6 pypi_0 pypi vhacdx 0.0.8.post1 pypi_0 pypi viser 0.2.3 pypi_0 pypi wandb 0.18.1 pypi_0 pypi wcwidth 0.2.13 pypi_0 pypi websocket-client 1.8.0 pypi_0 pypi websockets 12.0 pypi_0 pypi werkzeug 3.0.4 pypi_0 pypi wheel 0.44.0 py311h06a4308_0 xatlas 0.0.9 pypi_0 pypi xmod 1.8.1 pypi_0 pypi xxhash 3.5.0 pypi_0 pypi xz 5.4.6 h5eee18b_1 yaml 0.2.5 h7b6447c_0 yarl 1.11.1 pypi_0 pypi yourdfpy 0.0.56 pypi_0 pypi zipp 3.20.2 pypi_0 pypi zlib 1.2.13 h5eee18b_1 zstd 1.5.5 hc292b87_2 ```

yzslab commented 2 hours ago

Maybe jsonargparse==4.24 does not work on Python 3.11. Create a python 3.9 env then try again.

tayden commented 25 minutes ago

That did get things working

yzslab / gaussian-splatting-lightning

CLI parser doesn't correctly parse namespaced args #61