Additional fsspec tests

[joshmoore]

• [ ] Add test with --pre version of fsspec to detect upcoming issues
• [ ] Add reference filesystem test possibly by depending (optionally) on tifffile

Originally posted by @joshmoore in https://github.com/zarr-developers/zarr-python/issues/1324#issuecomment-1396725745

[jakirkham]

Thanks for raising Josh! Was thinking the same thing

That and/or setting up some kind of regression testing with fsspec. Something useful along these lines would be Numba's texasbbq project

[joshmoore]

That's nice! So in this case, @jakirkham, you'd create a zarr-integration-testing repo and add in tifffile or were you thinking an fsspec-integration-testing with zarr?

cc: @cgohlke @martindurant

[AkashS20]

Hi, I would like to contribute to this. I'm new to open source so if you could please guide me on solving this it will be of great help.

[forman]

@joshmoore @jakirkham

For example, FSMap of fsspec 2022.3.0 no longer has an attribute check (unfortunately without former deprecation), but current Zarr still uses it: https://github.com/zarr-developers/zarr-python/blob/main/zarr/storage.py#L148. @AkashS20, I guess, this is where regression testing applies, e.g., checking if fsspec interface are as expected.

Unfortunately, our latest xcube release is broken due to this:

EDIT

Unfortunately, our latest xcube ...

It's only broken when used used with fsspec >=2023.2.

[martindurant]

fsspec' FSMap does still have check=: https://github.com/fsspec/filesystem_spec/blob/master/fsspec/mapping.py#L37 and indeed it sets mapper.check a few lines later.

Furthermore, fsspec runs one simple xarray/zarr roundtrip (with s3 local mock) as part of its "downstream" CI job, which might catch this sort of thing.

[joshmoore]

@forman: what version of Zarr?

@martindurant: any idea on what the cause might be there then?

[martindurant]

No, I don't know, and I don't see a failure on xcube either :|