search

zarr-developers / zarr-python

An implementation of chunked, compressed, N-dimensional arrays for Python.
https://zarr.readthedocs.io
MIT License
1.52k stars 282 forks source link

Remove and investigate xfail added in #680 #681

Open Carreau opened 3 years ago

Carreau commented 3 years ago

680 marked a few tests as xfailed when migrating to a newer version of azurite; which apparently does not support empty string as a key.

We should investigate why, and if possible unmark those test from xfail.

Carreau commented 3 years ago

682 also remove some coverage we might want to re-enable.

Carreau commented 3 years ago

from https://docs.microsoft.com/en-us/rest/api/storageservices/naming-and-referencing-containers--blobs--and-metadata:

A blob name must be at least one character long and cannot be more than 1,024 characters long, for blobs in Azure Storage.

The Azure Storage emulator supports blob names up to 256 characters long. For more information, see Use the Azure storage emulator for development and testing.

So it looks like we need to ensure that in zarr, likely allow to read older blobs that might be empty, but prevent user to write new one that are to avoid losing access to them when Azure stop accepting those requests which right now likely are for backward compat.

Carreau commented 3 years ago

Ok, tracking things down it looks like the only case we query for an empty blob name is when we want the size of the full store; for which the correct path seem to be to list blobs.

Another way would be to use client.get_container_properties(....).properties.content_length which might be less correct if there are other things in the container, so improbable if the prefix used by zarr is empty.

I've made #686 which should fix tests; and we can update to use the above later on.