NCAS-CMS / cfdm

A Python reference implementation of the CF data model
http://ncas-cms.github.io/cfdm
MIT License
28 stars 11 forks source link

Reading of `http://` remote files does not work #262

Closed davidhassell closed 1 year ago

davidhassell commented 1 year ago

In cfdm 1.10.1.0 we get:

>>> remote_file = 'http://psl.noaa.gov/thredds/dodsC/Datasets/cru/crutem5/Monthlies/air.mon.anom.nobs.nc'
>>> import netCDF4
>>> nc = netCDF4.Dataset(remote_file, 'r')  # works
>>> import cfdm
>>> j = cfdm.read(remote_file)  # doesn't work
<snip>
packages/cfdm/read_write/netcdf/netcdfread.py:810, in NetCDFRead.read(self, filename, extra, default_version, external, extra_read_vars, _scan_only, verbose, mask, warnings, warn_valid, domain)
    807     raise IOError(f"Can't read directory {filename}")
    809 if not os.path.isfile(filename):
--> 810     raise IOError(f"Can't read non-existent file {filename}")
    812 g["filename"] = filename
    814 # ------------------------------------------------------------
    815 # Open the netCDF file to be read
    816 # ------------------------------------------------------------

OSError: Can't read non-existent file http://psl.noaa.gov/thredds/dodsC/Datasets/cru/crutem5/Monthlies/air.mon.anom.nobs.nc

The problem is that cfdm is testing remote_file for being a file with os.path.isfile, which returns False for remote URLs.

PR to follow.

davidhassell commented 1 year ago

Thanks to @bewithankit for spotting this!