search

euroargodev / argopy

A python library for Argo data beginners and experts
https://argopy.readthedocs.io
European Union Public License 1.2
176 stars 38 forks source link

Fix cache issue with ArgoIndex #346

Closed gmaze closed 4 months ago

gmaze commented 4 months ago

closes #345

gmaze commented 4 months ago

Managed to reproduce the error using only fsspec methods and raised the issue: https://github.com/fsspec/filesystem_spec/issues/1575

gmaze commented 4 months ago

I think I also found another issue leading to basically the same error:

When using a cached http fsspec store (a WholeFileCacheFileSystem on top of a HTTPFileSystem), then checking for the existence of a binary file fails if the parent web page does not link back to the existing binary file

This seems to be due to the fact that the HTTPFileSystem raises a UnicodeDecodeError when doing ls on a binary file, the list of links is empty and _ls_real return []