python-attrs / cattrs

Composable custom class converters for attrs, dataclasses and friends.
https://catt.rs
MIT License
820 stars 113 forks source link

Public API for Converter._structure_<collection>() #574

Closed sscherfke closed 2 months ago

sscherfke commented 2 months ago

Description

Typed Settings uses some of Converter's private structuring functions.

I know that I should not use them but they are too helpful. 🙈 Some changes in cattrs 24.8 broke TS (again). So I'd like to ask if it would make sense to make these functions (or a similar API) public. :-)

Tinche commented 2 months ago

Let's see if we can replace these with public versions.

What happens if you replace that code with:

collection_types = [
        # Order is important, tuple must be last!
        (is_sequence, converter.get_structure_hook(List)),
        (is_mutable_set, converter.get_structure_hook(Set)),
        (is_frozenset, converter.get_structure_hook(FrozenSet)),
        (is_tuple, converter.get_structure_hook(Tuple)),
    ]
sscherfke commented 2 months ago

This nearly works, but the hooks don't convert recursively, tests fail like this: E a: [['1', '2'], ['3', '4']] != [(1, 2), (3, 4)]

Tinche commented 2 months ago

I have an idea. I'm having trouble running nox in the typedsettings repo though.

❯ nox -e "test-3.12(latest_deps_version)"
nox > Running session test-3.12(latest_deps_version)
nox > Creating virtual environment (virtualenv) using python3.12 in .nox/test-3-12-latest_deps_version
nox > Package not found, running "build" ...
nox > git log -1 --pretty=%ct
nox > Setting SOURCE_DATE_EPOCH to 1724651936.
nox > python -m pip install build check-wheel-contents
nox > rm -rf dist
nox > python -m build --install=uv --outdir=dist
* Creating isolated environment: venv+uv...
* Using external uv from /home/tin/.local/bin/uv
* Installing packages in isolated environment:
  - hatch-vcs
  - hatchling>=1.5.0
* Getting build dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating isolated environment: venv+uv...
* Using external uv from /home/tin/.local/bin/uv
* Installing packages in isolated environment:
  - hatch-vcs
  - hatchling>=1.5.0
* Getting build dependencies for wheel...
* Building wheel...
Successfully built typed_settings-24.4.1.dev9.tar.gz and typed_settings-24.4.1.dev9-py3-none-any.whl
nox > check-wheel-contents dist
dist/typed_settings-24.4.1.dev9-py3-none-any.whl: OK
nox > Distribution contents:
📦 typed_settings-24.4.1.dev9-py3-none-any.whl
├── 📂 typed_settings
│   ├── 📄 __init__.py       2024-08-26T05-58-56Z 2180B
│   ├── 📄 _compat.py        2024-08-26T05-58-56Z 134B
│   ├── 📄 _core.py          2024-08-26T05-58-56Z 14663B
│   ├── 📄 _file_utils.py    2024-08-26T05-58-56Z 1659B
│   ├── 📄 _onepassword.py   2024-08-26T05-58-56Z 2138B
│   ├── 📄 argparse_utils.py 2024-08-26T05-58-56Z 254B
│   ├── 📄 attrs.py          2024-08-26T05-58-56Z 241B
│   ├── 📄 cli_argparse.py   2024-08-26T05-58-56Z 21780B
│   ├── 📄 cli_click.py      2024-08-26T05-58-56Z 23563B
│   ├── 📄 cli_utils.py      2024-08-26T05-58-56Z 15262B
│   ├── 📄 click_utils.py    2024-08-26T05-58-56Z 241B
│   ├── 📄 cls_attrs.py      2024-08-26T05-58-56Z 13518B
│   ├── 📄 cls_utils.py      2024-08-26T05-58-56Z 19731B
│   ├── 📄 constants.py      2024-08-26T05-58-56Z 466B
│   ├── 📄 converters.py     2024-08-26T05-58-56Z 36629B
│   ├── 📄 dict_utils.py     2024-08-26T05-58-56Z 6396B
│   ├── 📄 exceptions.py     2024-08-26T05-58-56Z 948B
│   ├── 📄 loaders.py        2024-08-26T05-58-56Z 18467B
│   ├── 📄 mypy.py           2024-08-26T05-58-56Z 1188B
│   ├── 📄 processors.py     2024-08-26T05-58-56Z 9924B
│   ├── 📄 py.typed          2024-08-26T05-58-56Z 0B
│   └── 📄 types.py          2024-08-26T05-58-56Z 8656B
└── 📂 typed_settings-24.4.1.dev9.dist-info
    ├── 📄 METADATA 2024-08-26T05-58-56Z 12427B
    ├── 📄 WHEEL    2024-08-26T05-58-56Z 87B
    ├── 📂 licenses
    │   └── 📄 LICENSE 2024-08-26T05-58-56Z 1072B
    └── 📄 RECORD   2024-08-26T05-58-56Z 2196B
📦 typed_settings-24.4.1.dev9.tar.gz
└── 📂 typed_settings-24.4.1.dev9
    ├── 📂 docs
    │   ├── 📄 Makefile           2024-08-26T05-58-56Z 637B
    │   ├── 📄 apiref.rst         2024-08-26T05-58-56Z 6832B
    │   ├── 📄 changelog.md       2024-08-26T05-58-56Z 53B
    │   ├── 📄 conf.py            2024-08-26T05-58-56Z 2740B
    │   ├── 📄 conftest.py        2024-08-26T05-58-56Z 6195B
    │   ├── 📄 development.md     2024-08-26T05-58-56Z 4920B
    │   ├── 📄 examples.md        2024-08-26T05-58-56Z 7042B
    │   ├── 📄 getting-started.md 2024-08-26T05-58-56Z 14076B
    │   ├── 📄 index.md           2024-08-26T05-58-56Z 3383B
    │   ├── 📄 license.md         2024-08-26T05-58-56Z 164B
    │   ├── 📄 why.md             2024-08-26T05-58-56Z 7564B
    │   ├── 📂 _static
    │   │   ├── 📄 1password-test-dark.png    2024-08-26T05-58-56Z 366071B
    │   │   ├── 📄 1password-test-light.png   2024-08-26T05-58-56Z 351000B
    │   │   ├── 📄 cli-argparse-dark.png      2024-08-26T05-58-56Z 387873B
    │   │   ├── 📄 cli-argparse-light.png     2024-08-26T05-58-56Z 361129B
    │   │   ├── 📄 cli-click-dark.png         2024-08-26T05-58-56Z 596177B
    │   │   ├── 📄 cli-click-light.png        2024-08-26T05-58-56Z 562046B
    │   │   ├── 📄 cli-rich_click-dark.png    2024-08-26T05-58-56Z 394639B
    │   │   ├── 📄 cli-rich_click-light.png   2024-08-26T05-58-56Z 373242B
    │   │   ├── 📄 custom.css                 2024-08-26T05-58-56Z 1176B
    │   │   ├── 📄 typed-settings-spacing.svg 2024-08-26T05-58-56Z 33208B
    │   │   ├── 📄 typed-settings.png         2024-08-26T05-58-56Z 36882B
    │   │   └── 📄 typed-settings.svg         2024-08-26T05-58-56Z 24668B
    │   ├── 📂 examples
    │   │   ├── 📂 black-pyproject.toml
    │   │   │   ├── 📄 black.py       2024-08-26T05-58-56Z 773B
    │   │   │   ├── 📄 pyproject.toml 2024-08-26T05-58-56Z 30B
    │   │   │   └── 📄 test.console   2024-08-26T05-58-56Z 542B
    │   │   ├── 📂 pypirc_0
    │   │   │   ├── 📄 pypirc.toml  2024-08-26T05-58-56Z 189B
    │   │   │   ├── 📄 pypirc_0.py  2024-08-26T05-58-56Z 461B
    │   │   │   └── 📄 test.console 2024-08-26T05-58-56Z 113B
    │   │   ├── 📂 pypirc_1
    │   │   │   ├── 📄 pypirc.toml  2024-08-26T05-58-56Z 175B
    │   │   │   ├── 📄 pypirc_1.py  2024-08-26T05-58-56Z 374B
    │   │   │   └── 📄 test.console 2024-08-26T05-58-56Z 113B
    │   │   ├── 📂 pytest-plugins
    │   │   │   ├── 📄 pytest.py    2024-08-26T05-58-56Z 1512B
    │   │   │   └── 📄 test.console 2024-08-26T05-58-56Z 835B
    │   │   └── 📂 python-gitlab
    │   │       ├── 📄 python-gitlab.toml 2024-08-26T05-58-56Z 213B
    │   │       ├── 📄 python_gitlab.py   2024-08-26T05-58-56Z 732B
    │   │       └── 📄 test.console       2024-08-26T05-58-56Z 168B
    │   └── 📂 guides
    │       ├── 📄 1password.md              2024-08-26T05-58-56Z 4734B
    │       ├── 📄 basic-settings-loading.md 2024-08-26T05-58-56Z 2897B
    │       ├── 📄 clis-argparse-or-click.md 2024-08-26T05-58-56Z 2089B
    │       ├── 📄 clis-argparse.md          2024-08-26T05-58-56Z 14310B
    │       ├── 📄 clis-click.md             2024-08-26T05-58-56Z 27564B
    │       ├── 📄 config-files.md           2024-08-26T05-58-56Z 13655B
    │       ├── 📄 environment-variables.md  2024-08-26T05-58-56Z 3677B
    │       ├── 📄 index.md                  2024-08-26T05-58-56Z 506B
    │       ├── 📄 post-processing.md        2024-08-26T05-58-56Z 5282B
    │       ├── 📄 settings-classes.md       2024-08-26T05-58-56Z 10588B
    │       └── 📄 writing-custom-loaders.md 2024-08-26T05-58-56Z 4750B
    ├── 📂 src
    │   └── 📂 typed_settings
    │       ├── 📄 __init__.py       2024-08-26T05-58-56Z 2180B
    │       ├── 📄 _compat.py        2024-08-26T05-58-56Z 134B
    │       ├── 📄 _core.py          2024-08-26T05-58-56Z 14663B
    │       ├── 📄 _file_utils.py    2024-08-26T05-58-56Z 1659B
    │       ├── 📄 _onepassword.py   2024-08-26T05-58-56Z 2138B
    │       ├── 📄 argparse_utils.py 2024-08-26T05-58-56Z 254B
    │       ├── 📄 attrs.py          2024-08-26T05-58-56Z 241B
    │       ├── 📄 cli_argparse.py   2024-08-26T05-58-56Z 21780B
    │       ├── 📄 cli_click.py      2024-08-26T05-58-56Z 23563B
    │       ├── 📄 cli_utils.py      2024-08-26T05-58-56Z 15262B
    │       ├── 📄 click_utils.py    2024-08-26T05-58-56Z 241B
    │       ├── 📄 cls_attrs.py      2024-08-26T05-58-56Z 13518B
    │       ├── 📄 cls_utils.py      2024-08-26T05-58-56Z 19731B
    │       ├── 📄 constants.py      2024-08-26T05-58-56Z 466B
    │       ├── 📄 converters.py     2024-08-26T05-58-56Z 36629B
    │       ├── 📄 dict_utils.py     2024-08-26T05-58-56Z 6396B
    │       ├── 📄 exceptions.py     2024-08-26T05-58-56Z 948B
    │       ├── 📄 loaders.py        2024-08-26T05-58-56Z 18467B
    │       ├── 📄 mypy.py           2024-08-26T05-58-56Z 1188B
    │       ├── 📄 processors.py     2024-08-26T05-58-56Z 9924B
    │       ├── 📄 py.typed          2024-08-26T05-58-56Z 0B
    │       └── 📄 types.py          2024-08-26T05-58-56Z 8656B
    ├── 📂 tests
    │   ├── 📄 __init__.py             2024-08-26T05-58-56Z 34B
    │   ├── 📄 conftest.py             2024-08-26T05-58-56Z 2831B
    │   ├── 📄 op                      2024-08-26T05-58-56Z 2319B
    │   ├── 📄 test_api.py             2024-08-26T05-58-56Z 1935B
    │   ├── 📄 test_cli_argparse.py    2024-08-26T05-58-56Z 8264B
    │   ├── 📄 test_cli_click.py       2024-08-26T05-58-56Z 29580B
    │   ├── 📄 test_cli_param_types.py 2024-08-26T05-58-56Z 30009B
    │   ├── 📄 test_cli_utils.py       2024-08-26T05-58-56Z 14375B
    │   ├── 📄 test_cls_attrs.py       2024-08-26T05-58-56Z 9891B
    │   ├── 📄 test_cls_utils.py       2024-08-26T05-58-56Z 23962B
    │   ├── 📄 test_converters.py      2024-08-26T05-58-56Z 29343B
    │   ├── 📄 test_core.py            2024-08-26T05-58-56Z 21577B
    │   ├── 📄 test_dict_utils.py      2024-08-26T05-58-56Z 6730B
    │   ├── 📄 test_file_utils.py      2024-08-26T05-58-56Z 3158B
    │   ├── 📄 test_loaders.py         2024-08-26T05-58-56Z 19243B
    │   ├── 📄 test_no_optionals.py    2024-08-26T05-58-56Z 2629B
    │   ├── 📄 test_onepassword.py     2024-08-26T05-58-56Z 3082B
    │   ├── 📄 test_processors.py      2024-08-26T05-58-56Z 12217B
    │   ├── 📄 test_readme.py          2024-08-26T05-58-56Z 3593B
    │   └── 📄 test_types.py           2024-08-26T05-58-56Z 4112B
    ├── 📄 .gitignore     2024-08-26T05-58-56Z 142B
    ├── 📄 LICENSE        2024-08-26T05-58-56Z 1072B
    ├── 📄 README.md      2024-08-26T05-58-56Z 7479B
    ├── 📄 pyproject.toml 2024-08-26T05-58-56Z 4876B
    └── 📄 PKG-INFO       2024-08-26T05-58-56Z 12427B
nox > Metadata:
    Metadata-Version: 2.3
    Name: typed-settings
    Version: 24.4.1.dev9
    Summary: Typed settings based on attrs classes
    Project-URL: Homepage, https://gitlab.com/sscherfke/typed-settings
    Project-URL: Documentation, https://typed-settings.readthedocs.io
    Project-URL: Changelog, https://typed-settings.readthedocs.io/en/latest/changelog.html
    Project-URL: Issues, https://gitlab.com/sscherfke/typed-settings/-/issues
    Project-URL: Source Code, https://gitlab.com/sscherfke/typed-settings
    Author-email: Stefan Scherfke <stefan@sofa-rockers.org>
    License-Expression: MIT
    License-File: LICENSE
    Keywords: configuration,options,settings,types,validation
    Classifier: Development Status :: 5 - Production/Stable
    Classifier: Environment :: Console
    Classifier: Intended Audience :: Developers
    Classifier: License :: OSI Approved :: MIT License
    Classifier: Natural Language :: English
    Classifier: Operating System :: OS Independent
    Classifier: Programming Language :: Python
    Classifier: Programming Language :: Python :: 3.8
    Classifier: Programming Language :: Python :: 3.9
    Classifier: Programming Language :: Python :: 3.10
    Classifier: Programming Language :: Python :: 3.11
    Classifier: Programming Language :: Python :: 3.12
    Classifier: Programming Language :: Python :: Implementation :: CPython
    Classifier: Programming Language :: Python :: Implementation :: PyPy
    Classifier: Topic :: Software Development :: Libraries :: Python Modules
    Requires-Python: >=3.8
    Requires-Dist: tomli>=2; python_version < '3.11'
    Provides-Extra: all
    Requires-Dist: attrs>=23.1; extra == 'all'
    Requires-Dist: cattrs>=22.2; extra == 'all'
    Requires-Dist: click-option-group; extra == 'all'
    Requires-Dist: click>=7; extra == 'all'
    Requires-Dist: jinja2; extra == 'all'
    Requires-Dist: pydantic>=2; extra == 'all'
    Provides-Extra: attrs
    Requires-Dist: attrs>=23.1; extra == 'attrs'
    Provides-Extra: cattrs
    Requires-Dist: cattrs>=22.2; extra == 'cattrs'
    Provides-Extra: click
    Requires-Dist: click>=7; extra == 'click'
    Provides-Extra: dev
    Requires-Dist: attrs>=23.1; extra == 'dev'
    Requires-Dist: black; extra == 'dev'
    Requires-Dist: cattrs>=22.2; extra == 'dev'
    Requires-Dist: click-option-group; extra == 'dev'
    Requires-Dist: click>=7; extra == 'dev'
    Requires-Dist: coverage>=5.3; extra == 'dev'
    Requires-Dist: furo>=2023.9; extra == 'dev'
    Requires-Dist: hypothesis; extra == 'dev'
    Requires-Dist: jinja2; extra == 'dev'
    Requires-Dist: mypy; extra == 'dev'
    Requires-Dist: myst-parser>=2.0; extra == 'dev'
    Requires-Dist: nox; extra == 'dev'
    Requires-Dist: pip-audit; extra == 'dev'
    Requires-Dist: pydantic>=2; extra == 'dev'
    Requires-Dist: pytest-cov; extra == 'dev'
    Requires-Dist: pytest>=7.2.0; extra == 'dev'
    Requires-Dist: rich-click>=1.6; extra == 'dev'
    Requires-Dist: ruff; extra == 'dev'
    Requires-Dist: sphinx-copybutton>=0.5.2; extra == 'dev'
    Requires-Dist: sphinx-inline-tabs>=2023.4.21; extra == 'dev'
    Requires-Dist: sphinx>=7.2; extra == 'dev'
    Requires-Dist: sybil>=6; extra == 'dev'
    Requires-Dist: types-toml; extra == 'dev'
    Requires-Dist: typing-extensions; extra == 'dev'
    Provides-Extra: docs
    Requires-Dist: attrs>=23.1; extra == 'docs'
    Requires-Dist: cattrs>=22.2; extra == 'docs'
    Requires-Dist: click-option-group; extra == 'docs'
    Requires-Dist: click>=7; extra == 'docs'
    Requires-Dist: furo>=2023.9; extra == 'docs'
    Requires-Dist: jinja2; extra == 'docs'
    Requires-Dist: myst-parser>=2.0; extra == 'docs'
    Requires-Dist: pydantic>=2; extra == 'docs'
    Requires-Dist: sphinx-copybutton>=0.5.2; extra == 'docs'
    Requires-Dist: sphinx-inline-tabs>=2023.4.21; extra == 'docs'
    Requires-Dist: sphinx>=7.2; extra == 'docs'
    Provides-Extra: jinja
    Requires-Dist: jinja2; extra == 'jinja'
    Provides-Extra: lint
    Requires-Dist: attrs>=23.1; extra == 'lint'
    Requires-Dist: black; extra == 'lint'
    Requires-Dist: cattrs>=22.2; extra == 'lint'
    Requires-Dist: click-option-group; extra == 'lint'
    Requires-Dist: click>=7; extra == 'lint'
    Requires-Dist: jinja2; extra == 'lint'
    Requires-Dist: mypy; extra == 'lint'
    Requires-Dist: pydantic>=2; extra == 'lint'
    Requires-Dist: ruff; extra == 'lint'
    Requires-Dist: types-toml; extra == 'lint'
    Provides-Extra: option-groups
    Requires-Dist: click-option-group; extra == 'option-groups'
    Requires-Dist: click>=7; extra == 'option-groups'
    Provides-Extra: pydantic
    Requires-Dist: pydantic>=2; extra == 'pydantic'
    Provides-Extra: test
    Requires-Dist: attrs>=23.1; extra == 'test'
    Requires-Dist: cattrs>=22.2; extra == 'test'
    Requires-Dist: click-option-group; extra == 'test'
    Requires-Dist: click>=7; extra == 'test'
    Requires-Dist: coverage>=5.3; extra == 'test'
    Requires-Dist: hypothesis; extra == 'test'
    Requires-Dist: jinja2; extra == 'test'
    Requires-Dist: pydantic>=2; extra == 'test'
    Requires-Dist: pytest-cov; extra == 'test'
    Requires-Dist: pytest>=7.2.0; extra == 'test'
    Requires-Dist: rich-click>=1.6; extra == 'test'
    Requires-Dist: sybil>=6; extra == 'test'
    Requires-Dist: typing-extensions; extra == 'test'
    Description-Content-Type: text/markdown

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                                                   Typed Settings                                                                                                                                                                                    ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

Load and merge settings from multiple different sources and present them in a structured, typed, and validated way!                                                                                                                                                                                                                                                                    

                                                                                                                                                                                         Why?                                                                                                                                                                                          

There are many different config file formats and libraries. Many of them have a narrow scope, don't integrate well with other libs, or lack in typing support.                                                                                                                                                                                                                         

Typed Settings' goal is to enable you to load settings from any source (e.g., env vars, config files, vaults) and can convert values to anything you need.                                                                                                                                                                                                                             

You can extend Typed Settings to support config sources that aren't supported yet and its extensive documentation will help you on your way.                                                                                                                                                                                                                                           

                                                                                                                                                                               What can it be used for?                                                                                                                                                                                

You can use Typed Settings in any context, e.g.:                                                                                                                                                                                                                                                                                                                                       

 • server processes                                                                                                                                                                                                                                                                                                                                                                    
 • containerized apps                                                                                                                                                                                                                                                                                                                                                                  
 • command line applications                                                                                                                                                                                                                                                                                                                                                           
 • scripts and tools for scientific experiments and data analysis                                                                                                                                                                                                                                                                                                                      

                                                                                                                                                                                   What does it do?                                                                                                                                                                                    

 • It loads settings from multiple sources (e.g., env vars, config files, secret vaults) in a unified way and merges the loaded values. You can add loaders for sources we cannot imagine yet.                                                                                                                                                                                         
 • It can post-process loaded values. This allows value interpolation/templating or calling helpers that retrieve secrets from vaults. You can create and add any processors you can image if the built-in ones are not enough.                                                                                                                                                        
 • You can add a CLI on top to let users update the loaded settings via command line arguments. Click and argparse are currently supported.                                                                                                                                                                                                                                            
 • Settings are cleanly structured and typed. The type annotations are used to convert the loaded settings to the proper types. This also includes higher level structures like dates, paths and various collections (lists, dicts, …). You can use attrs, dataclasses, or Pydantic to write settings classes.                                                                         
   Types Settings uses the powerful and fast cattrs) by default and falls back to an internal converter if cattrs is not installed.                                                                                                                                                                                                                                                    
 • No mandatory requirements.  Typed Settings works out-of-the box with dataclasses, argparse and its own converter.                                                                                                                                                                                                                                                                   

The documentation contains a full list of all features.                                                                                                                                                                                                                                                                                                                                

                                                                                                                                                                                     Installation                                                                                                                                                                                      

Install and update using pip:                                                                                                                                                                                                                                                                                                                                                          

 $ python -m pip install typed-settings                                                                                                                                                                                                                                                                                                                                                

Typed Settings as no required dependencies (except for tomli on older Python versions). You can install dependencies for optional features via                                                                                                                                                                                                                                         

 $ python -m pip install typed-settings[<feature>,...]                                                                                                                                                                                                                                                                                                                                 

Available features:                                                                                                                                                                                                                                                                                                                                                                    

 • typed-settings[attrs]: Enable settings classes via attrs.                                                                                                                                                                                                                                                                                                                           
 • typed-settings[pydantic]: Enable settings classes via Pydantic.                                                                                                                                                                                                                                                                                                                     
 • typed-settings[cattrs]: Enable usage of the powerful and fast cattrs converter.                                                                                                                                                                                                                                                                                                     
 • typed-settings[click]: Enable support for Click options.                                                                                                                                                                                                                                                                                                                            
 • typed-settings[option-groups]: Enable support for Click and Click option groups.                                                                                                                                                                                                                                                                                                    
 • typed-settings[jinja]: Enable support for value interpolation with Jinja templates.                                                                                                                                                                                                                                                                                                 
 • typed-settings[all]: Install all optional requirements.                                                                                                                                                                                                                                                                                                                             

                                                                                                                                                                                       Examples                                                                                                                                                                                        

                                                                                                                                                                            Hello, World!, with env. vars.                                                                                                                                                                             

This is a very simple example that demonstrates how you can load settings from environment variables.                                                                                                                                                                                                                                                                                  

 # example.py                                                                                                                                                                                                                                                                                                                                                                          
 import attrs                                                                                                                                                                                                                                                                                                                                                                          
 import typed_settings as ts                                                                                                                                                                                                                                                                                                                                                           

 @attrs.frozen                                                                                                                                                                                                                                                                                                                                                                         
 class Settings:                                                                                                                                                                                                                                                                                                                                                                       
     option: str                                                                                                                                                                                                                                                                                                                                                                       

 settings = ts.load(cls=Settings, appname="example")                                                                                                                                                                                                                                                                                                                                   
 print(settings)                                                                                                                                                                                                                                                                                                                                                                       

 $ EXAMPLE_OPTION="Hello, World!" python example.py                                                                                                                                                                                                                                                                                                                                    
 Settings(option='Hello, World!')                                                                                                                                                                                                                                                                                                                                                      

                                                                                                                                                                            Nested classes and config files                                                                                                                                                                            

Settings classes can be nested. Config files define a different section for each class.                                                                                                                                                                                                                                                                                                

 # example.py                                                                                                                                                                                                                                                                                                                                                                          
 import attrs                                                                                                                                                                                                                                                                                                                                                                          
 import click                                                                                                                                                                                                                                                                                                                                                                          

 import typed_settings as ts                                                                                                                                                                                                                                                                                                                                                           

 @attrs.frozen                                                                                                                                                                                                                                                                                                                                                                         
 class Host:                                                                                                                                                                                                                                                                                                                                                                           
     name: str                                                                                                                                                                                                                                                                                                                                                                         
     port: int                                                                                                                                                                                                                                                                                                                                                                         

 @attrs.frozen                                                                                                                                                                                                                                                                                                                                                                         
 class Settings:                                                                                                                                                                                                                                                                                                                                                                       
     host: Host                                                                                                                                                                                                                                                                                                                                                                        
     endpoint: str                                                                                                                                                                                                                                                                                                                                                                     
     retries: int = 3                                                                                                                                                                                                                                                                                                                                                                  

 settings = ts.load(                                                                                                                                                                                                                                                                                                                                                                   
     cls=Settings, appname="example", config_files=["settings.toml"]                                                                                                                                                                                                                                                                                                                   
 )                                                                                                                                                                                                                                                                                                                                                                                     
 print(settings)                                                                                                                                                                                                                                                                                                                                                                       

 # settings.toml                                                                                                                                                                                                                                                                                                                                                                       
 [example]                                                                                                                                                                                                                                                                                                                                                                             
 endpoint = "/spam"                                                                                                                                                                                                                                                                                                                                                                    

 [example.host]                                                                                                                                                                                                                                                                                                                                                                        
 name = "example.com"                                                                                                                                                                                                                                                                                                                                                                  
 port = 443                                                                                                                                                                                                                                                                                                                                                                            

 $ python example.py                                                                                                                                                                                                                                                                                                                                                                   
 Settings(host=Host(name='example.com', port=443), endpoint='/spam', retries=3)                                                                                                                                                                                                                                                                                                        

                                                                                                                                                                             Configurable settings loaders                                                                                                                                                                             

The first example used a convenience shortcut with pre-configured settings loaders. However, Typed Settings lets you explicitly configure which loaders are used and how they work:                                                                                                                                                                                                    

 # example.py                                                                                                                                                                                                                                                                                                                                                                          
 import attrs                                                                                                                                                                                                                                                                                                                                                                          
 import typed_settings as ts                                                                                                                                                                                                                                                                                                                                                           

 @attrs.frozen                                                                                                                                                                                                                                                                                                                                                                         
 class Settings:                                                                                                                                                                                                                                                                                                                                                                       
     option: str                                                                                                                                                                                                                                                                                                                                                                       

 settings = ts.load_settings(                                                                                                                                                                                                                                                                                                                                                          
     cls=Settings,                                                                                                                                                                                                                                                                                                                                                                     
     loaders=[                                                                                                                                                                                                                                                                                                                                                                         
         ts.FileLoader(                                                                                                                                                                                                                                                                                                                                                                
             files=[],                                                                                                                                                                                                                                                                                                                                                                 
             env_var="EXAMPLE_SETTINGS",                                                                                                                                                                                                                                                                                                                                               
             formats={                                                                                                                                                                                                                                                                                                                                                                 
                 "*.toml": ts.TomlFormat("example"),                                                                                                                                                                                                                                                                                                                                   
             },                                                                                                                                                                                                                                                                                                                                                                        
         ),                                                                                                                                                                                                                                                                                                                                                                            
         ts.EnvLoader(prefix="EXAMPLE_"),                                                                                                                                                                                                                                                                                                                                              
       ],                                                                                                                                                                                                                                                                                                                                                                              
 )                                                                                                                                                                                                                                                                                                                                                                                     
 print(settings)                                                                                                                                                                                                                                                                                                                                                                       

 $ EXAMPLE_OPTION="Hello, World!" python example.py                                                                                                                                                                                                                                                                                                                                    
 Settings(option='Hello, World!')                                                                                                                                                                                                                                                                                                                                                      

In order to write your own loaders or support new file formats, you need to implement the Loader or FileFormat protocols.                                                                                                                                                                                                                                                              

You can also pass a custom cattrs converter to add support for additional Python types.                                                                                                                                                                                                                                                                                                

                                                                                                                                                                                Command Line Interfaces                                                                                                                                                                                

Typed Settings can generate a command line interfaces (CLI) based on your settings. These CLIs will load settings as described above and let users override the loades settings with command line argumments.                                                                                                                                                                          

Typed Settings supports argparse and click.                                                                                                                                                                                                                                                                                                                                            

                                                                                                                                                                                       Argparse                                                                                                                                                                                        

 # example.py                                                                                                                                                                                                                                                                                                                                                                          
 import attrs                                                                                                                                                                                                                                                                                                                                                                          
 import typed_settings as ts                                                                                                                                                                                                                                                                                                                                                           

 @attrs.frozen                                                                                                                                                                                                                                                                                                                                                                         
 class Settings:                                                                                                                                                                                                                                                                                                                                                                       
     a_str: str = ts.option(default="default", help="A string")                                                                                                                                                                                                                                                                                                                        
     an_int: int = ts.option(default=3, help="An int")                                                                                                                                                                                                                                                                                                                                 

 @ts.cli(Settings, "example")                                                                                                                                                                                                                                                                                                                                                          
 def main(settings):                                                                                                                                                                                                                                                                                                                                                                   
     print(settings)                                                                                                                                                                                                                                                                                                                                                                   

 if __name__ == "__main__":                                                                                                                                                                                                                                                                                                                                                            
     main()                                                                                                                                                                                                                                                                                                                                                                            

 $ python example.py --help                                                                                                                                                                                                                                                                                                                                                            
 usage: example.py [-h] [--a-str TEXT] [--an-int INT]                                                                                                                                                                                                                                                                                                                                  

 options:                                                                                                                                                                                                                                                                                                                                                                              
   -h, --help    show this help message and exit                                                                                                                                                                                                                                                                                                                                       

 Settings:                                                                                                                                                                                                                                                                                                                                                                             
   Settings options                                                                                                                                                                                                                                                                                                                                                                    

   --a-str TEXT  A string [default: default]                                                                                                                                                                                                                                                                                                                                           
   --an-int INT  An int [default: 3]                                                                                                                                                                                                                                                                                                                                                   
 $ python example.py --a-str=spam --an-int=1                                                                                                                                                                                                                                                                                                                                           
 Settings(a_str='spam', an_int=1)                                                                                                                                                                                                                                                                                                                                                      

                                                                                                                                                                                         Click                                                                                                                                                                                         

 # example.py                                                                                                                                                                                                                                                                                                                                                                          
 import attrs                                                                                                                                                                                                                                                                                                                                                                          
 import click                                                                                                                                                                                                                                                                                                                                                                          
 import typed_settings as ts                                                                                                                                                                                                                                                                                                                                                           

 @attrs.frozen                                                                                                                                                                                                                                                                                                                                                                         
 class Settings:                                                                                                                                                                                                                                                                                                                                                                       
     a_str: str = ts.option(default="default", help="A string")                                                                                                                                                                                                                                                                                                                        
     an_int: int = ts.option(default=3, help="An int")                                                                                                                                                                                                                                                                                                                                 

 @click.command()                                                                                                                                                                                                                                                                                                                                                                      
 @ts.click_options(Settings, "example")                                                                                                                                                                                                                                                                                                                                                
 def main(settings):                                                                                                                                                                                                                                                                                                                                                                   
     print(settings)                                                                                                                                                                                                                                                                                                                                                                   

 if __name__ == "__main__":                                                                                                                                                                                                                                                                                                                                                            
     main()                                                                                                                                                                                                                                                                                                                                                                            

 $ python example.py --help                                                                                                                                                                                                                                                                                                                                                            
 Usage: example.py [OPTIONS]                                                                                                                                                                                                                                                                                                                                                           

 Options:                                                                                                                                                                                                                                                                                                                                                                              
   --a-str TEXT      A string  [default: default]                                                                                                                                                                                                                                                                                                                                      
   --an-int INTEGER  An int  [default: 3]                                                                                                                                                                                                                                                                                                                                              
   --help            Show this message and exit.                                                                                                                                                                                                                                                                                                                                       
 $ python example.py --a-str=spam --an-int=1                                                                                                                                                                                                                                                                                                                                           
 Settings(a_str='spam', an_int=1)                                                                                                                                                                                                                                                                                                                                                      

                                                                                                                                                                                     Project Links                                                                                                                                                                                     

 • Changelog                                                                                                                                                                                                                                                                                                                                                                           
 • Documentation                                                                                                                                                                                                                                                                                                                                                                       
 • GitLab                                                                                                                                                                                                                                                                                                                                                                              
 • Issues and Bugs                                                                                                                                                                                                                                                                                                                                                                     
 • PyPI                                                                                                                                                                                                                                                                                                                                                                                
nox > python -m pip install 'typed-settings[test] @ dist/typed_settings-24.4.1.dev9-py3-none-any.whl'
nox > Command python -m pip install 'typed-settings[test] @ dist/typed_settings-24.4.1.dev9-py3-none-any.whl' failed with exit code 1:
Collecting typed-settings@ dist/typed_settings-24.4.1.dev9-py3-none-any.whl (from typed-settings[test]@ dist/typed_settings-24.4.1.dev9-py3-none-any.whl)
ERROR: Could not install packages due to an OSError: Invalid URL 'dist/typed_settings-24.4.1.dev9-py3-none-any.whl': No scheme supplied. Perhaps you meant https://dist/typed_settings-24.4.1.dev9-py3-none-any.whl?

[notice] A new release of pip is available: 24.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
nox > Session test-3.12(latest_deps_version) failed.

Any idea of how to proceed?

Tinche commented 2 months ago

I think I figured it out: https://gitlab.com/sscherfke/typed-settings/-/merge_requests/40