search

aai-institute / lakefs-spec

An fsspec implementation for the lakeFS project
http://lakefs-spec.org/
Apache License 2.0
39 stars 4 forks source link

Gate functionality on lakeFS backend version #208

Closed nicholasjng closed 9 months ago

nicholasjng commented 9 months ago

Motivation

Follow-up of #201: We found out that fs.touch() will not work for any lakeFS server <1.3.1, since the backend had trouble with missing MIME types (see also https://github.com/treeverse/lakeFS/issues/7127).

We should raise an informative error in case any file system functionality is broken/unsupported for a backend version.

Requested solution

Load the client's VersionConfig (https://pydocs-sdk.lakefs.io/docs/InternalApi.html#get_lake_fs_version), parse it out, check against a minimum version and maybe raise NotImplementedErrors if the server version is too old.

Bonus: Cache the API response by host, so that multiple client requests can reuse the obtained value.