search

nextGEMS / catalog

Intake catalog for nextgems
5 stars 7 forks source link

IFS parquet references: explicitly state remote_protocol #63

Closed d70-t closed 8 months ago

d70-t commented 8 months ago

The reference filesystem implementation needs to know in advance which (fsspec-) protocol(s) is used to access data chunks in referenced files. There is a heuristic in place, but this heuristic apparently fails in some cases. As we know that the references are (currently) backed by files on the filessystem, we can just fix the protocol to "file" to work around this issue (and possible improve performance slightly).

trackow commented 8 months ago

Thank you for fixing this so quickly. Merged!