Command line parsing of *.json values

LourencoVazPato commented 2 years ago

🐛 Bug report

Not sure if a bug or a feature, but when I call a script (e.g. PyTorch-Lightning CLI) with an argument like --dataset_path *.json, the parser reads the json file and interprets it as a configuration file (not as a dataset file in this case), and errors out because it is not a valid config file.

I can see there's documentation on parsing file paths, but cannot find any reference on reading them as string arguments.

Is it possible to disable this parsing? What other alternatives are there?

Thanks in advance

To reproduce

Create a data.json file containing some JSON dataset. E.g.:

{"columns":["id","name"],"data":[[0,"John"],[1,"Mary"]]}

import os
from typing import List, Union, Optional

import pandas as pd
from jsonargparse import ArgumentParser

PathLike = Union[str, os.PathLike]

class DataModule:
    def __init__(
        self, data_path: Optional[Union[pd.DataFrame, List[PathLike], PathLike]] = None
    ):
        self.dataset = data_path

class SubDataModule(DataModule):
    def __int__(
        self, data_path: Optional[Union[pd.DataFrame, List[PathLike], PathLike]] = None
    ):
        super().__init__(data_path)

parser = ArgumentParser()

# as initialized in pytorch_lightning.utilities.cli.add_lightning_class_args
parser.add_subclass_arguments(DataModule, "data", fail_untyped=False, required=True)

cfg = parser.parse_args(
    ["--data.class_path=SubDataModule", "--data.data_path=data.json"]
)
print(cfg)

cls = parser.instantiate_classes(cfg)
print(cls)

Errors out with:

usage: jsonarg.py [-h] [--data.help CLASS_PATH_OR_NAME]
                  --data CONFIG | CLASS_PATH_OR_NAME | .INIT_ARG_NAME VALUE
jsonarg.py: error: Parser key "data": Problem with given class_path "__main__.SubDataModule":
  - Configuration check failed :: Parser key "data_path": Value "{'columns': ['id', 'name'], 'data': [[0, 'John'], [1, 'Mary']]}" does not validate against any of the types in typing.Union[typing.List[typing.Union[str, os.PathLike]], str, os.PathLike, NoneType]:
    - Expected a List but got "{'columns': ['id', 'name'], 'data': [[0, 'John'], [1, 'Mary']]}"
    - Expected a <class 'str'> but got "{'columns': ['id', 'name'], 'data': [[0, 'John'], [1, 'Mary']]}"
    - Type <class 'os.PathLike'> expects: a class path (str); or a dict with a class_path entry; or a dict with init_args (if class path given previously). Got "{'columns': ['id', 'name'], 'data': [[0, 'John'], [1, 'Mary']]}".
    - Expected a <class 'NoneType'> but got "{'columns': ['id', 'name'], 'data': [[0, 'John'], [1, 'Mary']]}"

Curiously, it passes if I use a data.csv instead of JSON.

Expected behavior

The parser reads the arg value as a str.

Environment

jsonargparse version (e.g., 4.8.0): 4.13.0
Python version (e.g., 3.9): 3.9.1
pytorch-lightning version: 1.6.5
How jsonargparse was installed (e.g. pip install jsonargparse[all]): poetry add jsonargparse
OS (e.g., Linux): macOS, and Linux

mauvilsa commented 2 years ago

The parsing depends on the type hints. What is the type hint for dataset_path? If it is str, after parsing the value should be a string with whatever you have in the command line. If the type is a path like the docs you linked, the parsed value will not be str but a Path object.

mauvilsa commented 2 years ago

If you don't have type hints, then add them. This is how the parser knows how to validate. In LightningCLI it is configured such that when there is no type hint, it defaults to Any. If the type is Any then the parser does not know that it should be the path to a json file or its contents.

LourencoVazPato commented 2 years ago

In my LightningDataModule, the type hint for dataset is

dataframe_or_data_path: Optional[
    Union[pd.DataFrame, List[PathLike], PathLike]
] = None

where PathLike = Union[str, os.PathLike].

Could this be an issue?

mauvilsa commented 2 years ago

With the following:

import os
from jsonargparse import ArgumentParser
from typing import Any, List, Union

parser = ArgumentParser()
PathLike = Union[str, os.PathLike]
parser.add_argument('--path', type=Union[List[PathLike], PathLike])
cfg = parser.parse_args(['--path=issue_159.json'])
print(cfg)

The result is correct Namespace(path='issue_159.json').

os.PathLike is not currently supported. But since the union has str there is no issue in that case. Support for os.PathLike can be added. But still this does not explain what you originally described. You will need to post a minimal reproducible script.

LourencoVazPato commented 2 years ago

@mauvilsa I updated the description with the code to reproduce. I have initialized the arguments with .add_subclass_arguments() as in pytorch-lightning v1.6.5.

mauvilsa commented 2 years ago

@LourencoVazPato thank you for adding the reproduction code. Unfortunately I have been unable to reproduce it. First I tried in a normal virtual environment and then I tried with poetry to be a close as possible to what you reported. In Ubuntu 20.04 I get:

$ poetry run python issue_159.py 
The currently activated Python version 3.8.10 is not supported by the project (^3.9).
Trying to find and use a compatible version. 
Using python3.9 (3.9.13)
Namespace(data=Namespace(class_path='__main__.SubDataModule', init_args=Namespace(data_path='data.json')))
Namespace(data=<__main__.SubDataModule object at 0x7f7c5f185c70>)

The pyproject.toml is the following:

[tool.poetry]
name = "issue-159"
version = "0.1.0"
description = ""
authors = []
readme = "README.md"
packages = [{include = "issue_159"}]

[tool.poetry.dependencies]
python = "^3.9"
jsonargparse = "4.13.0"
pandas = "^1.4.4"
pytorch-lightning = "1.6.5"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

LourencoVazPato commented 2 years ago

@mauvilsa I've been able to reproduce with this poetry env:

[tool.poetry]
name = "issue-159"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.9"
jsonargparse = {extras = ["signatures"], version = "4.13.0"}
pandas = "^1.4.4"
pytorch-lightning = "1.6.5"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

If you want to reproduce my python env here's the poetry lock.

poetry.lock

```lock [[package]] name = "absl-py" version = "1.2.0" description = "Abseil Python Common Libraries, see https://github.com/abseil/abseil-py." category = "main" optional = false python-versions = ">=3.6" [[package]] name = "aiohttp" version = "3.8.1" description = "Async http client/server framework (asyncio)" category = "main" optional = false python-versions = ">=3.6" [package.dependencies] aiosignal = ">=1.1.2" async-timeout = ">=4.0.0a3,<5.0" attrs = ">=17.3.0" charset-normalizer = ">=2.0,<3.0" frozenlist = ">=1.1.1" multidict = ">=4.5,<7.0" yarl = ">=1.0,<2.0" [package.extras] speedups = ["aiodns", "brotli", "cchardet"] [[package]] name = "aiosignal" version = "1.2.0" description = "aiosignal: a list of registered asynchronous callbacks" category = "main" optional = false python-versions = ">=3.6" [package.dependencies] frozenlist = ">=1.1.0" [[package]] name = "async-timeout" version = "4.0.2" description = "Timeout context manager for asyncio programs" category = "main" optional = false python-versions = ">=3.6" [[package]] name = "attrs" version = "22.1.0" description = "Classes Without Boilerplate" category = "main" optional = false python-versions = ">=3.5" [package.extras] dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"] docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"] tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"] tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"] [[package]] name = "cachetools" version = "5.2.0" description = "Extensible memoizing collections and decorators" category = "main" optional = false python-versions = "~=3.7" [[package]] name = "certifi" version = "2022.6.15" description = "Python package for providing Mozilla's CA Bundle." category = "main" optional = false python-versions = ">=3.6" [[package]] name = "charset-normalizer" version = "2.1.1" description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet." category = "main" optional = false python-versions = ">=3.6.0" [package.extras] unicode_backport = ["unicodedata2"] [[package]] name = "colorama" version = "0.4.5" description = "Cross-platform colored terminal text." category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" [[package]] name = "frozenlist" version = "1.3.1" description = "A list-like structure which implements collections.abc.MutableSequence" category = "main" optional = false python-versions = ">=3.7" [[package]] name = "fsspec" version = "2022.8.2" description = "File-system specification" category = "main" optional = false python-versions = ">=3.7" [package.dependencies] aiohttp = {version = "<4.0.0a0 || >4.0.0a0,<4.0.0a1 || >4.0.0a1", optional = true, markers = "extra == \"http\""} requests = {version = "*", optional = true, markers = "extra == \"http\""} [package.extras] abfs = ["adlfs"] adl = ["adlfs"] arrow = ["pyarrow (>=1)"] dask = ["dask", "distributed"] dropbox = ["dropboxdrivefs", "requests", "dropbox"] entrypoints = ["importlib-metadata"] fuse = ["fusepy"] gcs = ["gcsfs"] git = ["pygit2"] github = ["requests"] gs = ["gcsfs"] gui = ["panel"] hdfs = ["pyarrow (>=1)"] http = ["requests", "aiohttp (!=4.0.0a0,!=4.0.0a1)"] libarchive = ["libarchive-c"] oci = ["ocifs"] s3 = ["s3fs"] sftp = ["paramiko"] smb = ["smbprotocol"] ssh = ["paramiko"] tqdm = ["tqdm"] [[package]] name = "google-auth" version = "2.11.0" description = "Google Authentication Library" category = "main" optional = false python-versions = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*" [package.dependencies] cachetools = ">=2.0.0,<6.0" pyasn1-modules = ">=0.2.1" rsa = {version = ">=3.1.4,<5", markers = "python_version >= \"3.6\""} six = ">=1.9.0" [package.extras] aiohttp = ["requests (>=2.20.0,<3.0.0dev)", "aiohttp (>=3.6.2,<4.0.0dev)"] enterprise_cert = ["cryptography (==36.0.2)", "pyopenssl (==22.0.0)"] pyopenssl = ["pyopenssl (>=20.0.0)"] reauth = ["pyu2f (>=0.1.5)"] [[package]] name = "google-auth-oauthlib" version = "0.4.6" description = "Google Authentication Library" category = "main" optional = false python-versions = ">=3.6" [package.dependencies] google-auth = ">=1.0.0" requests-oauthlib = ">=0.7.0" [package.extras] tool = ["click (>=6.0.0)"] [[package]] name = "grpcio" version = "1.48.1" description = "HTTP/2-based RPC framework" category = "main" optional = false python-versions = ">=3.6" [package.dependencies] six = ">=1.5.2" [package.extras] protobuf = ["grpcio-tools (>=1.48.1)"] [[package]] name = "idna" version = "3.3" description = "Internationalized Domain Names in Applications (IDNA)" category = "main" optional = false python-versions = ">=3.5" [[package]] name = "importlib-metadata" version = "4.12.0" description = "Read metadata from Python packages" category = "main" optional = false python-versions = ">=3.7" [package.dependencies] zipp = ">=0.5" [package.extras] docs = ["sphinx", "jaraco.packaging (>=9)", "rst.linker (>=1.9)"] perf = ["ipython"] testing = ["pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.3)", "packaging", "pyfakefs", "flufl.flake8", "pytest-perf (>=0.9.2)", "pytest-black (>=0.3.7)", "pytest-mypy (>=0.9.1)", "importlib-resources (>=1.3)"] [[package]] name = "jsonargparse" version = "4.13.0" description = "Parsing of command line options, yaml/jsonnet config files and/or environment variables based on argparse." category = "main" optional = false python-versions = ">=3.6" [package.dependencies] PyYAML = ">=3.13" [package.extras] all = ["docstring-parser (>=0.7.3)", "jsonschema (>=3.2.0)", "validators (>=0.14.2)", "requests (>=2.18.4)", "fsspec (>=0.8.4)", "argcomplete (>=2.0.0)", "ruyaml (>=0.20.0)", "omegaconf (>=2.1.1)", "reconplogger (>=4.4.0)", "jsonnet-binary (>=0.17.0)", "jsonnet (>=0.13.0)", "typing-extensions (>=3.10.0.0)"] argcomplete = ["argcomplete (>=2.0.0)"] dev = ["coverage (>=4.5.1)", "responses (>=0.12.0)", "Sphinx (>=1.7.9)", "sphinx-rtd-theme (>=0.4.3)", "autodocsumm (>=0.1.10)", "sphinx-autodoc-typehints (>=1.11.1)", "pre-commit (>=2.19.0)", "pylint (>=1.8.3)", "pycodestyle (>=2.5.0)", "mypy (>=0.701)", "tox (>=3.25.0)"] doc = ["Sphinx (>=1.7.9)", "sphinx-rtd-theme (>=0.4.3)", "autodocsumm (>=0.1.10)", "sphinx-autodoc-typehints (>=1.11.1)"] fsspec = ["fsspec (>=0.8.4)"] jsonnet = ["jsonnet-binary (>=0.17.0)", "jsonnet (>=0.13.0)"] jsonschema = ["jsonschema (>=3.2.0)"] maintainer = ["bump2version (>=0.5.11)", "twine (>=4.0.0)"] omegaconf = ["omegaconf (>=2.1.1)"] reconplogger = ["reconplogger (>=4.4.0)"] ruyaml = ["ruyaml (>=0.20.0)"] signatures = ["docstring-parser (>=0.7.3)"] test = ["coverage (>=4.5.1)", "responses (>=0.12.0)"] test_no_urls = ["coverage (>=4.5.1)"] typing_extensions = ["typing-extensions (>=3.10.0.0)"] urls = ["validators (>=0.14.2)", "requests (>=2.18.4)"] [[package]] name = "markdown" version = "3.4.1" description = "Python implementation of Markdown." category = "main" optional = false python-versions = ">=3.7" [package.dependencies] importlib-metadata = {version = ">=4.4", markers = "python_version < \"3.10\""} [package.extras] testing = ["coverage", "pyyaml"] [[package]] name = "markupsafe" version = "2.1.1" description = "Safely add untrusted strings to HTML/XML markup." category = "main" optional = false python-versions = ">=3.7" [[package]] name = "multidict" version = "6.0.2" description = "multidict implementation" category = "main" optional = false python-versions = ">=3.7" [[package]] name = "numpy" version = "1.23.2" description = "NumPy is the fundamental package for array computing with Python." category = "main" optional = false python-versions = ">=3.8" [[package]] name = "oauthlib" version = "3.2.0" description = "A generic, spec-compliant, thorough implementation of the OAuth request-signing logic" category = "main" optional = false python-versions = ">=3.6" [package.extras] rsa = ["cryptography (>=3.0.0)"] signals = ["blinker (>=1.4.0)"] signedtoken = ["cryptography (>=3.0.0)", "pyjwt (>=2.0.0,<3)"] [[package]] name = "packaging" version = "21.3" description = "Core utilities for Python packages" category = "main" optional = false python-versions = ">=3.6" [package.dependencies] pyparsing = ">=2.0.2,<3.0.5 || >3.0.5" [[package]] name = "pandas" version = "1.4.4" description = "Powerful data structures for data analysis, time series, and statistics" category = "main" optional = false python-versions = ">=3.8" [package.dependencies] numpy = [ {version = ">=1.18.5", markers = "platform_machine != \"aarch64\" and platform_machine != \"arm64\" and python_version < \"3.10\""}, {version = ">=1.19.2", markers = "platform_machine == \"aarch64\" and python_version < \"3.10\""}, {version = ">=1.20.0", markers = "platform_machine == \"arm64\" and python_version < \"3.10\""}, {version = ">=1.21.0", markers = "python_version >= \"3.10\""}, ] python-dateutil = ">=2.8.1" pytz = ">=2020.1" [package.extras] test = ["hypothesis (>=5.5.3)", "pytest (>=6.0)", "pytest-xdist (>=1.31)"] [[package]] name = "protobuf" version = "3.19.4" description = "Protocol Buffers" category = "main" optional = false python-versions = ">=3.5" [[package]] name = "pyasn1" version = "0.4.8" description = "ASN.1 types and codecs" category = "main" optional = false python-versions = "*" [[package]] name = "pyasn1-modules" version = "0.2.8" description = "A collection of ASN.1-based protocols modules." category = "main" optional = false python-versions = "*" [package.dependencies] pyasn1 = ">=0.4.6,<0.5.0" [[package]] name = "pydeprecate" version = "0.3.2" description = "Deprecation tooling" category = "main" optional = false python-versions = ">=3.6" [[package]] name = "pyparsing" version = "3.0.9" description = "pyparsing module - Classes and methods to define and execute parsing grammars" category = "main" optional = false python-versions = ">=3.6.8" [package.extras] diagrams = ["railroad-diagrams", "jinja2"] [[package]] name = "python-dateutil" version = "2.8.2" description = "Extensions to the standard Python datetime module" category = "main" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7" [package.dependencies] six = ">=1.5" [[package]] name = "pytorch-lightning" version = "1.6.5" description = "PyTorch Lightning is the lightweight PyTorch wrapper for ML researchers. Scale your models. Write less boilerplate." category = "main" optional = false python-versions = ">=3.7" [package.dependencies] fsspec = {version = ">=2021.05.0,<2021.06.0 || >2021.06.0", extras = ["http"]} numpy = ">=1.17.2" packaging = ">=17.0" protobuf = "<=3.20.1" pyDeprecate = ">=0.3.1" PyYAML = ">=5.4" tensorboard = ">=2.2.0" torch = ">=1.8" torchmetrics = ">=0.4.1" tqdm = ">=4.57.0" typing-extensions = ">=4.0.0" [package.extras] all = ["matplotlib (>3.1)", "torchtext (>=0.9)", "omegaconf (>=2.0.5)", "hydra-core (>=1.0.5)", "jsonargparse[signatures] (>=4.7.1)", "gcsfs (>=2021.5.0)", "rich (>=10.2.2,!=10.15.0.a)", "neptune-client (>=0.10.0)", "comet-ml (>=3.1.12)", "mlflow (>=1.0.0)", "test-tube (>=0.7.5)", "wandb (>=0.8.21)", "coverage (>=6.4)", "codecov (>=2.1)", "pytest (>=6.0)", "pytest-rerunfailures (>=10.2)", "mypy (>=0.920)", "flake8 (>=3.9.2)", "pre-commit (>=1.0)", "pytest-forked", "cloudpickle (>=1.3)", "scikit-learn (>0.22.1)", "onnxruntime", "pandas", "torchvision (>=0.9)", "gym[classic_control] (>=0.17.0)", "ipython", "fairscale (>=0.4.5)", "deepspeed", "horovod (>=0.21.2,!=0.24.0)", "hivemind (>=1.0.1)"] deepspeed = ["deepspeed"] dev = ["matplotlib (>3.1)", "torchtext (>=0.9)", "omegaconf (>=2.0.5)", "hydra-core (>=1.0.5)", "jsonargparse[signatures] (>=4.7.1)", "gcsfs (>=2021.5.0)", "rich (>=10.2.2,!=10.15.0.a)", "neptune-client (>=0.10.0)", "comet-ml (>=3.1.12)", "mlflow (>=1.0.0)", "test-tube (>=0.7.5)", "wandb (>=0.8.21)", "coverage (>=6.4)", "codecov (>=2.1)", "pytest (>=6.0)", "pytest-rerunfailures (>=10.2)", "mypy (>=0.920)", "flake8 (>=3.9.2)", "pre-commit (>=1.0)", "pytest-forked", "cloudpickle (>=1.3)", "scikit-learn (>0.22.1)", "onnxruntime", "pandas"] examples = ["torchvision (>=0.9)", "gym[classic_control] (>=0.17.0)", "ipython"] extra = ["matplotlib (>3.1)", "torchtext (>=0.9)", "omegaconf (>=2.0.5)", "hydra-core (>=1.0.5)", "jsonargparse[signatures] (>=4.7.1)", "gcsfs (>=2021.5.0)", "rich (>=10.2.2,!=10.15.0.a)"] fairscale = ["fairscale (>=0.4.5)"] hivemind = ["hivemind (>=1.0.1)"] horovod = ["horovod (>=0.21.2,!=0.24.0)"] loggers = ["neptune-client (>=0.10.0)", "comet-ml (>=3.1.12)", "mlflow (>=1.0.0)", "test-tube (>=0.7.5)", "wandb (>=0.8.21)"] strategies = ["fairscale (>=0.4.5)", "deepspeed", "horovod (>=0.21.2,!=0.24.0)", "hivemind (>=1.0.1)"] test = ["coverage (>=6.4)", "codecov (>=2.1)", "pytest (>=6.0)", "pytest-rerunfailures (>=10.2)", "mypy (>=0.920)", "flake8 (>=3.9.2)", "pre-commit (>=1.0)", "pytest-forked", "cloudpickle (>=1.3)", "scikit-learn (>0.22.1)", "onnxruntime", "pandas"] [[package]] name = "pytz" version = "2022.2.1" description = "World timezone definitions, modern and historical" category = "main" optional = false python-versions = "*" [[package]] name = "pyyaml" version = "6.0" description = "YAML parser and emitter for Python" category = "main" optional = false python-versions = ">=3.6" [[package]] name = "requests" version = "2.28.1" description = "Python HTTP for Humans." category = "main" optional = false python-versions = ">=3.7, <4" [package.dependencies] certifi = ">=2017.4.17" charset-normalizer = ">=2,<3" idna = ">=2.5,<4" urllib3 = ">=1.21.1,<1.27" [package.extras] socks = ["PySocks (>=1.5.6,!=1.5.7)"] use_chardet_on_py3 = ["chardet (>=3.0.2,<6)"] [[package]] name = "requests-oauthlib" version = "1.3.1" description = "OAuthlib authentication support for Requests." category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" [package.dependencies] oauthlib = ">=3.0.0" requests = ">=2.0.0" [package.extras] rsa = ["oauthlib[signedtoken] (>=3.0.0)"] [[package]] name = "rsa" version = "4.9" description = "Pure-Python RSA implementation" category = "main" optional = false python-versions = ">=3.6,<4" [package.dependencies] pyasn1 = ">=0.1.3" [[package]] name = "six" version = "1.16.0" description = "Python 2 and 3 compatibility utilities" category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*" [[package]] name = "tensorboard" version = "2.10.0" description = "TensorBoard lets you watch Tensors Flow" category = "main" optional = false python-versions = ">=3.6" [package.dependencies] absl-py = ">=0.4" google-auth = ">=1.6.3,<3" google-auth-oauthlib = ">=0.4.1,<0.5" grpcio = ">=1.24.3" markdown = ">=2.6.8" numpy = ">=1.12.0" protobuf = ">=3.9.2,<3.20" requests = ">=2.21.0,<3" tensorboard-data-server = ">=0.6.0,<0.7.0" tensorboard-plugin-wit = ">=1.6.0" werkzeug = ">=1.0.1" [[package]] name = "tensorboard-data-server" version = "0.6.1" description = "Fast data loading for TensorBoard" category = "main" optional = false python-versions = ">=3.6" [[package]] name = "tensorboard-plugin-wit" version = "1.8.1" description = "What-If Tool TensorBoard plugin." category = "main" optional = false python-versions = "*" [[package]] name = "torch" version = "1.12.1" description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" category = "main" optional = false python-versions = ">=3.7.0" [package.dependencies] typing-extensions = "*" [[package]] name = "torchmetrics" version = "0.9.3" description = "PyTorch native Metrics" category = "main" optional = false python-versions = ">=3.7" [package.dependencies] numpy = ">=1.17.2" packaging = "*" torch = ">=1.3.1" [package.extras] test = ["pycocotools", "codecov (>=2.1)", "check-manifest", "torch-complex", "scikit-learn (>1.0,<1.1.1)", "cloudpickle (>=1.3)", "mir-eval (>=0.6)", "pytest-cov (>2.10)", "pytest-timeout", "scikit-image (>0.17.1)", "pytest (>=6.0.0,<7.0.0)", "pypesq", "transformers (>=4.0)", "pre-commit (>=1.0)", "fire", "bert-score (==0.3.10)", "pytorch-msssim", "psutil", "coverage (>5.2)", "huggingface-hub (<0.7)", "pytest-doctestplus (>=0.9.0)", "requests", "rouge-score (==0.0.4)", "jiwer (>=2.3.0)", "twine (>=3.2)", "sacrebleu (>=2.0.0)", "fast-bss-eval (>=0.1.0)", "phmdoctest (>=1.1.1)", "mypy (>=0.790)"] all = ["pystoi", "pycocotools", "torchvision (>=0.8)", "sphinx (>=4.0,<5.0)", "sphinxcontrib-fulltoc (>=1.0)", "sphinxcontrib-mockautodoc", "sphinx-autodoc-typehints (>=1.0)", "sphinx-copybutton (>=0.3)", "docutils (>=0.16)", "pandoc (>=1.0)", "nbsphinx (>=0.8)", "sphinx-togglebutton (>=0.2)", "sphinx-paramlinks (>=0.5.1)", "myst-parser", "lpips", "torch-fidelity", "scipy", "torchvision", "pytorch-lightning (>=1.5)", "codecov (>=2.1)", "check-manifest", "torch-complex", "scikit-learn (>1.0,<1.1.1)", "cloudpickle (>=1.3)", "mir-eval (>=0.6)", "pytest-cov (>2.10)", "pytest-timeout", "scikit-image (>0.17.1)", "pytest (>=6.0.0,<7.0.0)", "pypesq", "transformers (>=4.0)", "pre-commit (>=1.0)", "fire", "bert-score (==0.3.10)", "pytorch-msssim", "psutil", "coverage (>5.2)", "huggingface-hub (<0.7)", "pytest-doctestplus (>=0.9.0)", "requests", "rouge-score (==0.0.4)", "jiwer (>=2.3.0)", "twine (>=3.2)", "sacrebleu (>=2.0.0)", "fast-bss-eval (>=0.1.0)", "phmdoctest (>=1.1.1)", "mypy (>=0.790)", "nltk (>=3.6)", "regex (>=2021.9.24)", "tqdm (>=4.41.0)"] audio = ["pystoi"] detection = ["pycocotools", "torchvision (>=0.8)"] docs = ["sphinx (>=4.0,<5.0)", "sphinxcontrib-fulltoc (>=1.0)", "sphinxcontrib-mockautodoc", "sphinx-autodoc-typehints (>=1.0)", "sphinx-copybutton (>=0.3)", "docutils (>=0.16)", "pandoc (>=1.0)", "nbsphinx (>=0.8)", "sphinx-togglebutton (>=0.2)", "sphinx-paramlinks (>=0.5.1)", "myst-parser"] image = ["lpips", "torch-fidelity", "scipy", "torchvision"] integrate = ["pytorch-lightning (>=1.5)"] text = ["nltk (>=3.6)", "regex (>=2021.9.24)", "tqdm (>=4.41.0)"] [[package]] name = "tqdm" version = "4.64.1" description = "Fast, Extensible Progress Meter" category = "main" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7" [package.dependencies] colorama = {version = "*", markers = "platform_system == \"Windows\""} [package.extras] dev = ["py-make (>=0.1.0)", "twine", "wheel"] notebook = ["ipywidgets (>=6)"] slack = ["slack-sdk"] telegram = ["requests"] [[package]] name = "typing-extensions" version = "4.3.0" description = "Backported and Experimental Type Hints for Python 3.7+" category = "main" optional = false python-versions = ">=3.7" [[package]] name = "urllib3" version = "1.26.12" description = "HTTP library with thread-safe connection pooling, file post, and more." category = "main" optional = false python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4" [package.extras] brotli = ["brotlicffi (>=0.8.0)", "brotli (>=1.0.9)", "brotlipy (>=0.6.0)"] secure = ["pyOpenSSL (>=0.14)", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "certifi", "urllib3-secure-extra", "ipaddress"] socks = ["PySocks (>=1.5.6,!=1.5.7,<2.0)"] [[package]] name = "werkzeug" version = "2.2.2" description = "The comprehensive WSGI web application library." category = "main" optional = false python-versions = ">=3.7" [package.dependencies] MarkupSafe = ">=2.1.1" [package.extras] watchdog = ["watchdog"] [[package]] name = "yarl" version = "1.8.1" description = "Yet another URL library" category = "main" optional = false python-versions = ">=3.7" [package.dependencies] idna = ">=2.0" multidict = ">=4.0" [[package]] name = "zipp" version = "3.8.1" description = "Backport of pathlib-compatible object wrapper for zip files" category = "main" optional = false python-versions = ">=3.7" [package.extras] docs = ["sphinx", "jaraco.packaging (>=9)", "rst.linker (>=1.9)", "jaraco.tidelift (>=1.4)"] testing = ["pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.3)", "jaraco.itertools", "func-timeout", "pytest-black (>=0.3.7)", "pytest-mypy (>=0.9.1)"] [metadata] lock-version = "1.1" python-versions = "^3.9" content-hash = "1971b60cf23cc5eb6e306a91740b0f5934ecb15b68cc5ec8314cbb86d2c0cf59" [metadata.files] absl-py = [] aiohttp = [] aiosignal = [] async-timeout = [] attrs = [] cachetools = [] certifi = [] charset-normalizer = [] colorama = [] frozenlist = [] fsspec = [] google-auth = [] google-auth-oauthlib = [] grpcio = [] idna = [] importlib-metadata = [] jsonargparse = [] markdown = [] markupsafe = [] multidict = [] numpy = [] oauthlib = [] packaging = [] pandas = [] protobuf = [] pyasn1 = [] pyasn1-modules = [] pydeprecate = [] pyparsing = [] python-dateutil = [] pytorch-lightning = [] pytz = [] pyyaml = [] requests = [] requests-oauthlib = [] rsa = [] six = [] tensorboard = [] tensorboard-data-server = [] tensorboard-plugin-wit = [] torch = [] torchmetrics = [] tqdm = [] typing-extensions = [] urllib3 = [] werkzeug = [] yarl = [] zipp = [] ```

mauvilsa commented 2 years ago

I had a silly mistake when trying to reproduce. I was able to reproduce it in a normal virtual environment and any python version. I have pushed the fix in commit https://github.com/omni-us/jsonargparse/commit/744c0c1105a02a3dc3845d6488189cb22ca684ee.

omni-us / jsonargparse