fetch_forecast fails silently

jamesthesken commented 1 year ago

Looking into the examples, I'm having a tough time getting fetch_forecast.py running successfully.

My environment:

Python 3.11.3
requests==2.31.0
urllib3==2.0.3
matplotlib==3.7.2
numpy==1.25.0

Running

$ python3 fetch_forecast.py

Fetching GFS Wave Data
Failed to fetch wave forecast data

Some steps I took to debug:

Cloned the grippy repository and installed in my conda environment.

I came across a warning

Failed to download https://noaa-gfs-bdp-pds.s3.amazonaws.com/gfs.20230713/18/wave/gridded/gfswave.t18z.atlocn.0p16.f000.grib2: Retry.__init__() got an unexpected keyword argument 'method_whitelist'

Which prompted me to upgrade requests and urllib3 to the latest versions and modify tools.py:

# method_whitelist deprecated in favor of allowed_methods
def retry_session(retries=1):
    session = requests.Session()
    retries = Retry(total=retries,
                backoff_factor=0.1,
                status_forcelist=[500, 502, 503, 504],
                allowed_methods=frozenset(['GET', 'POST']))

    session.mount('https://', HTTPAdapter(max_retries=retries))
    session.mount('http://', HTTPAdapter(max_retries=retries))
    return session

What package versions are working for you? I would love to help document and assist with this project where possible.

Thank you for your help.

pheidler commented 1 year ago

I found this: https://stackoverflow.com/questions/28904607/cannot-import-requests-packages-urllib3-util-retry

Importing directly from urllib3.util fixed fetching data for me, but now I'm down the rabbit hole of trying to parse the grib data with pygrib. Any advice?

amandabaughan commented 2 months ago

Did you end up figuring it out? I am now also trying to parse grib data...

mpiannucci commented 2 months ago

So pygrib should be the default now in this package. Things are a little outdated here because i have moved on to use other methods, but everything here should work except if urrllib needs to be updated. Happy to help others get sorted if they are having issues!

pheidler commented 2 months ago

@amandabaughan It's been a while since I've worked with this library, but if I remember correctly, I couldn't get pygrib to run natively on Windows. I believe I used a VM running Ubuntu or Docker to get everything working.

mpiannucci commented 2 months ago

These days, most people Are using xarray and cfgrib to do this kind of workflow

mpiannucci / surfpy

fetch_forecast fails silently #20