Closed imanshafiei540 closed 5 months ago
@imanshafiei540 this is really weird because RIO_TILER_MAX_THREADS
doesn't conflict with TITILER_API_
prefix 🤷
Okay, I tried to dig more into the issue and ended up here in this line from Pydantic==2.6.4
:
self.__pydantic_validator__.validate_python(data, self_instance=self)
Somehow, the api_settings = ApiSettings()
coming from src/titiler/application/titiler/application/main.py
cannot be validated by __pydantic_validator__
and I'm not sure what's the reason.
Could be some issues between Pydantic
and Pydantic-Settings.
I hope someone else can also verify the issue; I'm constantly thinking maybe I'm doing something wrong on my end.
Anyway, thank you for the quick response, and I'll try to figure out what's wrong; it's probably time to raise an issue on the Pydantic-Setiings repo.
can you share the full log
? and/or share a reproducible example 🙏
Sure, I'll prepare something ASAP, just as a note, everything is working with pydantic-settings==2.1.0
2.2.0
or above will break the TiTiler.
Okay, here is how to reproduce (it's actually pretty simple):
venv
)zsh
)
pip install pre-commit -e src/titiler/core\[test\] -e src/titiler/extensions\[test,cogeo,stac\] -e src/titiler/mosaic\[test\] -e src/titiler/application\[test\]
python -m pytest src/titiler/application --cov=titiler.application --cov-report=xml --cov-append --cov-report=term-missing
.env
file in the root of the project with the content below (it's not specified to this env variable, you can use other env variables if you want to):
RIO_TILER_MAX_THREADS=50
python -m pytest src/titiler/application --cov=titiler.application --cov-report=xml --cov-append --cov-report=term-missing
pydantic_core._pydantic_core.ValidationError: 1 validation error for ApiSettings
More context (pip freeze
):
Python=3.11.8
affine==2.4.0
annotated-types==0.6.0
anyio==4.3.0
attrs==23.2.0
boto3==1.34.66
botocore==1.34.66
brotlipy==0.7.0
cachetools==5.3.3
certifi==2024.2.2
cffi==1.16.0
cfgv==3.4.0
click==8.1.7
click-plugins==1.1.1
cligj==0.7.2
cogeo-mosaic==7.1.0
color-operations==0.1.3
coverage==7.4.4
cramjam==2.6.2
distlib==0.3.8
fastapi==0.110.0
filelock==3.13.1
geojson-pydantic==1.0.2
h11==0.14.0
httpcore==1.0.4
httpx==0.27.0
identify==2.5.35
idna==3.6
iniconfig==2.0.0
Jinja2==3.1.3
jmespath==1.0.1
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
MarkupSafe==2.1.5
morecantile==5.3.0
nodeenv==1.8.0
numexpr==2.9.0
numpy==1.26.4
packaging==24.0
platformdirs==4.2.0
pluggy==1.4.0
pre-commit==3.6.2
pycparser==2.21
pydantic==2.6.4
pydantic-settings==2.2.1
pydantic_core==2.16.3
pyparsing==3.1.2
pyproj==3.6.1
pystac==1.9.0
pytest==8.1.1
pytest-asyncio==0.23.6
pytest-cov==4.1.0
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
PyYAML==6.0.1
rasterio==1.3.9
referencing==0.34.0
rio-cogeo==5.2.0
rio-stac==0.8.1
rio-tiler==6.4.1
rpds-py==0.18.0
s3transfer==0.10.1
shapely==2.0.3
simplejson==3.19.2
six==1.16.0
sniffio==1.3.1
snuggs==1.4.7
starlette==0.36.3
starlette-cramjam==0.3.2
supermorecado==0.1.2
-e git+ssh://git@github.com/developmentseed/titiler.git@ffd67af34c2807a6e1447817f943446a58441ed8#egg=titiler.application&subdirectory=src/titiler/application
-e git+ssh://git@github.com/developmentseed/titiler.git@ffd67af34c2807a6e1447817f943446a58441ed8#egg=titiler.core&subdirectory=src/titiler/core
-e git+ssh://git@github.com/developmentseed/titiler.git@ffd67af34c2807a6e1447817f943446a58441ed8#egg=titiler.extensions&subdirectory=src/titiler/extensions
-e git+ssh://git@github.com/developmentseed/titiler.git@ffd67af34c2807a6e1447817f943446a58441ed8#egg=titiler.mosaic&subdirectory=src/titiler/mosaic
typing_extensions==4.10.0
urllib3==2.2.1
virtualenv==20.25.1
uvicorn
to reproduce the issueAfter creating the .env
file:
uvicorn
by pip install uvicorn
uvicorn titiler.application.main:app --workers 3 --host localhost --port 8002
Now you should be able to see the errors, by removing the .env
file, everything is working properly.
Let me know if you need more information. Thank you.
ok I've been able to reproduce this
uvicorn titiler.application.main:app --port 8000 --reload
INFO: Will watch for changes in these directories: ['/Users/vincentsarago/Dev/DevSeed/titiler']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [79319] using WatchFiles
Process SpawnProcess-1:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.9/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/opt/homebrew/Cellar/python@3.9/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/uvicorn/server.py", line 60, in run
return asyncio.run(self.serve(sockets=sockets))
File "/opt/homebrew/Cellar/python@3.9/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/uvicorn/server.py", line 67, in serve
config.load()
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/uvicorn/config.py", line 479, in load
self.loaded_app = import_from_string(self.app)
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
File "/opt/homebrew/Cellar/python@3.9/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/Users/vincentsarago/Dev/DevSeed/titiler/src/titiler/application/titiler/application/main.py", line 51, in <module>
api_settings = ApiSettings()
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/pydantic_settings/main.py", line 84, in __init__
super().__init__(
File "/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/pydantic/main.py", line 171, in __init__
self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ApiSettings
rio_tiler_max_threads
Extra inputs are not permitted [type=extra_forbidden, input_value='50', input_type=str]
For further information visit https://errors.pydantic.dev/2.6/v/extra_forbidden
This is really weird and I wonder if this is a pydantic-setting bug instead of a titiler one.
pydantic ticket: https://github.com/pydantic/pydantic-settings/issues/245
I would be happy to review any PR to add extra='ignore'
👍
Thanks for confirming the issue. I'll try to create a PR as soon as possible.
Problem description
I encountered an error related to the
rio_tiler_max_threads
environment variable when using TiTiler with the following environment configuration:.env
file:pyproject.toml
file:pip freeze
output:The error occurred with the current implementation:
The error message is:
Expected Output
No errors when loading the environment variables.
Proposed Fix
I fixed the issue by allowing extra settings in the
SettingsConfigDict
:Environment Information