meteostat / meteostat-python

Access and analyze historical weather and climate data with Python.
https://dev.meteostat.net/python/
MIT License
428 stars 60 forks source link

The mechanism for setting cache_dir could be more elegant by the use of environment variables #107

Closed dnk8n closed 4 months ago

dnk8n commented 2 years ago

Despite doing the following:

from meteostat import Daily, Point
import tempfile

Daily.cache_dir = tempfile.mkdtemp()

I still receive errors from elsewhere in the code:

[ERROR] OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'
Traceback (most recent call last):
  File "/var/task/fleet_reb.py", line 365, in lambda_handler_predict
    predictor_input = get_model_inputs(
  File "/var/task/***/***/***/model.py", line 256, in get_model_inputs
    weater_data = Daily(Point(59.3887844, 17.762308), date_start, date_end)
  File "/var/lang/lib/python3.8/site-packages/meteostat/interface/daily.py", line 98, in __init__
    self._init_time_series(loc, start, end, model, flags)
  File "/var/lang/lib/python3.8/site-packages/meteostat/interface/timeseries.py", line 156, in _init_time_series
    stations = loc.get_stations("daily", start, end, model)
  File "/var/lang/lib/python3.8/site-packages/meteostat/interface/point.py", line 76, in get_stations
    stations = Stations()
  File "/var/lang/lib/python3.8/site-packages/meteostat/interface/stations.py", line 107, in __init__
    self._load()
  File "/var/lang/lib/python3.8/site-packages/meteostat/interface/stations.py", line 82, in _load
    if self.max_age > 0 and file_in_cache(path, self.max_age):
  File "/var/lang/lib/python3.8/site-packages/meteostat/core/cache.py", line 38, in file_in_cache
    os.makedirs(directory)
  File "/var/lang/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/var/lang/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/var/lang/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/var/lang/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)

It seems Stations is called indirectly. Do I need to import the class just to set cache_dir?

Is there not an easier way, i.e. setting an environment variable which applies to all?

In this case I am using AWS Lambda and home directory is not writeable.

clampr commented 2 years ago

I agree. There probably is a better way. For now you'll have to set the cache_dir on the class manually.

freekwiekmeijer commented 1 year ago

Problem reproduces, solution is to set cache dir on Daily and Stations classes separately. Agree that an environment variable like METEOSTAT_CACHE_DIR would be the preferred solution.

clampr commented 4 months ago

Will be considered for version 2.