zarr-developers / VirtualiZarr

Create virtual Zarr stores from archival data files using xarray syntax
https://virtualizarr.readthedocs.io/en/latest/
Apache License 2.0
90 stars 16 forks source link

open_virtual_dataset HTTPS access fails #195

Closed ayushnag closed 1 month ago

ayushnag commented 1 month ago

Currently open_virtual_dataset cannot open a dataset when given a public HTTPS url. This happens because reader_options is set on this line to s3-like syntax which probably messes up the https request. The functionality is correct in _fsspec_openfile_from_filepath so it's a pretty simple fix to just update the linked line in xarray.py

Also noticed by @kthyng in https://github.com/zarr-developers/VirtualiZarr/pull/186#issuecomment-2231450332

Here is an example:

>>> from virtualizarr import open_virtual_dataset
>>> open_virtual_dataset("https://github.com/pydata/xarray-data/raw/master/ROMS_example.nc", indexes={})
Traceback (most recent call last): ...
FileNotFoundError: https://github.com/pydata/xarray-data/raw/master/ROMS_example.nc
>>> import fsspec
>>> z = fsspec.filesystem("https").open("https://github.com/pydata/xarray-data/raw/master/ROMS_example.nc")
>>> z.readline()
>>> b'\x89HDF\r\n'
kthyng commented 1 month ago

Ah, thanks for noticing!