search

ome / ome-zarr-py

Implementation of next-generation file format (NGFF) specifications for storing bioimaging data in the cloud.
https://pypi.org/project/ome-zarr
Other
152 stars 54 forks source link

[Reader] reading HCS file on AWS S3 results in error #321

Closed elyall closed 12 months ago

elyall commented 1 year ago

When trying to read an HCS file on AWS S3 (e.g. s3://bucket/file.ome.zarr) the reader returns Exception: Could not find first well. Reading a specific well from the HCS file works fine (e.g. s3://bucket/file.ome.zarr/A/2/0).

The issue seems to be here where self.zarr.create(self.well_paths[0]) :

Steps to reproduce

  1. install terraform
  2. generate an AWS bucket
    code

Create a new directory and save the following to main.tf:

# ------------------------------------------------------------------------------
# Variables
# ------------------------------------------------------------------------------

variable "region" {
  default     = "us-west-2"
  description = "AWS region"
  type        = string
}

variable "LOCAL_AWS_USER" {
  description = "The name of the AWS account to allow writing to the bucket"
  type        = string
}

# ------------------------------------------------------------------------------
# Versions
# ------------------------------------------------------------------------------

terraform {
  required_version = ">= 1.0.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.72"
    }
    random = {
      source  = "hashicorp/random"
      version = "3.1.0" # Replace with the appropriate version of the random provider
    }
  }
}

# ------------------------------------------------------------------------------
# Providers
# ------------------------------------------------------------------------------

provider "aws" {
  region = var.region
}

# ------------------------------------------------------------------------------
# Public S3 bucket
# ------------------------------------------------------------------------------

resource "random_string" "example" {
  length  = 8
  special = false
  lower   = true
  upper   = false
}

resource "aws_s3_bucket" "example" {
  bucket = "test-bucket-${random_string.example.result}"

  tags = {
    Name        = "Test bucket"
    Environment = "Dev"
  }
}

resource "aws_s3_bucket_ownership_controls" "example" {
  bucket = aws_s3_bucket.example.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

resource "aws_s3_bucket_public_access_block" "example" {
  bucket = aws_s3_bucket.example.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

resource "aws_s3_bucket_acl" "example" {
  depends_on = [
    aws_s3_bucket_ownership_controls.example,
    aws_s3_bucket_public_access_block.example,
  ]

  bucket = aws_s3_bucket.example.id
  acl    = "public-read"
}

# ------------------------------------------------------------------------------
# Access policies
# ------------------------------------------------------------------------------

resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = aws_s3_bucket.example.id
  policy = jsonencode(
    {
      "Version" : "2012-10-17",
      "Statement" : [
        {
          "Sid" : "PublicReadGetObject",
          "Effect" : "Allow",
          "Principal" : "*",
          "Action" : "s3:GetObject",
          "Resource" : "arn:aws:s3:::${aws_s3_bucket.example.id}/*"
        }
      ]
    }
  )
}

resource "aws_iam_policy" "example_put" {
  name        = "${aws_s3_bucket.example.bucket}-put"
  path        = "/"
  description = "Policy to allow users to put objects in s3 bucket ${aws_s3_bucket.example.bucket}"

  policy = <<-EOT
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Upload",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": [
                "${aws_s3_bucket.example.arn}",
                "${aws_s3_bucket.example.arn}/*"
            ]
        }
    ]
}
EOT
}

resource "aws_iam_user_policy_attachment" "example_put" {
  user       = var.LOCAL_AWS_USER
  policy_arn = aws_iam_policy.example_put.arn
}

Update the two variables then run in the terminal:

terraform init
terraform apply -auto-approve

Later to destroy the infrastructure run:

terraform destroy -auto-approve

  1. write an HCS ome-zarr file to the bucket
    code

Update path to reflect the name of the bucket created then execute the following code to generate the file:

path = "s3://test-bucket-XXXXXXXX/test.ome.zarr"

import numpy as np
import zarr
from ome_zarr.io import parse_url
from ome_zarr.writer import write_image, write_plate_metadata, write_well_metadata

row_names = ["A", "B"]
col_names = ["1", "2", "3"]
well_paths = ["A/2", "B/3"]
field_paths = ["0", "1", "2"]

# generate data
mean_val=10
num_wells = len(well_paths)
num_fields = len(field_paths)
size_xy = 128
size_z = 10
rng = np.random.default_rng(0)
data = rng.poisson(mean_val, size=(num_wells, num_fields, size_z, size_xy, size_xy)).astype(np.uint8)

# write the plate of images and corresponding metadata
store = parse_url(path, mode="w").store
root = zarr.group(store=store)
write_plate_metadata(root, row_names, col_names, well_paths)
for wi, wp in enumerate(well_paths):
    row, col = wp.split("/")
    row_group = root.require_group(row)
    well_group = row_group.require_group(col)
    write_well_metadata(well_group, field_paths)
    for fi, field in enumerate(field_paths):
        image_group = well_group.require_group(str(field))
        write_image(image=data[wi, fi], group=image_group, axes="zyx", storage_options=dict(chunks=(1, size_xy, size_xy)))

  1. try reading it from the bucket
    code
import napari

viewer = napari.Viewer()
viewer.open(str(path), plugin="napari-ome-zarr")

Error

full error ```shell --------------------------------------------------------------------------- Exception Traceback (most recent call last) File [napari/components/viewer_model.py:1092], in ViewerModel.open(self, path, stack, plugin, layer_type, **kwargs) 1089 _path = [_path] if not isinstance(_path, list) else _path 1090 if plugin: 1091 added.extend( -> 1092 self._add_layers_with_plugins( 1093 _path, 1094 kwargs=kwargs, 1095 plugin=plugin, 1096 layer_type=layer_type, 1097 stack=_stack, 1098 ) 1099 ) 1100 # no plugin choice was made 1101 else: 1102 layers = self._open_or_raise_error( 1103 _path, kwargs, layer_type, _stack 1104 ) File [napari/components/viewer_model.py:1292], in ViewerModel._add_layers_with_plugins(self, paths, stack, kwargs, plugin, layer_type) 1290 else: 1291 assert len(paths) == 1 -> 1292 layer_data, hookimpl = read_data_with_plugins( 1293 paths, plugin=plugin, stack=stack 1294 ) 1296 # glean layer names from filename. These will be used as *fallback* 1297 # names, if the plugin does not return a name kwarg in their meta dict. 1298 filenames = [] File [napari/plugins/io.py:77], in read_data_with_plugins(paths, plugin, stack) 74 assert len(paths) == 1 75 hookimpl: Optional[HookImplementation] ---> 77 res = _npe2.read(paths, plugin, stack=stack) 78 if res is not None: 79 _ld, hookimpl = res File [napari/plugins/_npe2.py:63], in read(paths, plugin, stack) 61 npe1_path = paths[0] 62 try: ---> 63 layer_data, reader = io_utils.read_get_reader( 64 npe1_path, plugin_name=plugin 65 ) 66 except ValueError as e: 67 # plugin wasn't passed and no reader was found 68 if 'No readers returned data' not in str(e): File [npe2/io_utils.py:66], in read_get_reader(path, plugin_name, stack) 62 if stack is None: 63 # "npe1" old path 64 # Napari 0.4.15 and older, hopefully we can drop this and make stack mandatory 65 new_path, new_stack = v1_to_v2(path) ---> 66 return _read( 67 new_path, plugin_name=plugin_name, return_reader=True, stack=new_stack 68 ) 69 else: 70 assert isinstance(path, list) File [npe2/io_utils.py:165], in _read(paths, stack, plugin_name, return_reader, _pm) 160 read_func = rdr.exec( 161 kwargs={"path": paths, "stack": stack, "_registry": _pm.commands} 162 ) 163 if read_func is not None: 164 # if the reader function raises an exception here, we don't try to catch it --> 165 if layer_data := read_func(paths, stack=stack): 166 return (layer_data, rdr) if return_reader else layer_data 168 if plugin_name: File [npe2/manifest/contributions/_readers.py:60], in ReaderContribution.exec..npe1_compat(paths, stack) 57 @wraps(callable_) 58 def npe1_compat(paths, *, stack): 59 path = v2_to_v1(paths, stack) ---> 60 return callable_(path) File [napari_ome_zarr/_reader.py:105], in transform..f(*args, **kwargs) 102 def f(*args: Any, **kwargs: Any) -> List[LayerData]: 103 results: List[LayerData] = list() --> 105 for node in nodes: 106 data: List[Any] = node.data 107 metadata: Dict[str, Any] = {} File [ome-zarr-py/ome_zarr/reader.py:621], in Reader.__call__(self) 620 def __call__(self) -> Iterator[Node]: --> 621 node = Node(self.zarr, self) 622 if node.specs: # Something has matched 623 LOGGER.debug("treating %s as ome-zarr", self.zarr) File [ome-zarr-py/ome_zarr/reader.py:58], in Node.__init__(self, zarr, root, visibility, plate_labels) 56 self.specs.append(PlateLabels(self)) 57 elif Plate.matches(zarr): ---> 58 self.specs.append(Plate(self)) 59 # self.add(zarr, plate_labels=True) 60 if Well.matches(zarr): File [ome-zarr-py/ome_zarr/reader.py:473], in Plate.__init__(self, node) 471 super().__init__(node) 472 LOGGER.debug("Plate created with ZarrLocation fmt: %s", self.zarr.fmt) --> 473 self.get_pyramid_lazy(node) File [ome-zarr-py/ome_zarr/reader.py:498], in Plate.get_pyramid_lazy(self, node) 496 well_spec: Optional[Well] = well_node.first(Well) 497 if well_spec is None: --> 498 raise Exception("Could not find first well") 499 self.first_field_path = well_spec.well_data["images"][0]["path"] 500 self.numpy_type = well_spec.numpy_type Exception: Could not find first well ```

Versions tested

ome-zarr = 0.8.2 and 0.8.3.dev0 napari-ome-zarr = 0.5.2 python = 3.10.12

Fyi, I first mentioned this error here.

will-moore commented 1 year ago

Thanks for the detailed report. Unfortunately I don't know how to reproduce this as all the remote s3 buckets I have access to are via https, e.g This works OK:

napari --plugin napari-ome-zarr https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0001A/2551.zarr

We have access to a minio server, but I don't know how to replicate your workflow there. I see you're using terraform but I guess that needs more than a local install on my Mac?

elyall commented 1 year ago

A local install of terraform paired with an AWS account would work fine. But to make life easier I've temporarily made a publicly readable bucket and saved an HCS dataset to an ome-zarr file there. Please try using this file for testing: s3://test-bucket-xbahruvc/test.ome.zarr.

To generate this file I saved skimage.data.cells3d to path A/2/0. No other wells exist in the dataset defined as a 96 well plate.

elyall commented 1 year ago

The issue is in ome_zarr.io.ZarrLocation.subpath which can't handle FSStore.fs.protocol == ["s3","s3a"]. It treats the S3 path as a url by passing it to urllib.parse.urljoin when it should be treated like a file path. This also means that Google Cloud Storage paths and Azure Blob Storage paths would also fail to be handled correctly. There's a couple simple ways to deal with this:

  1. Add the dependency cloudpathlib which has a AnyPath class that wraps both pathlib.Path and cloudpathlib.CloudPath (which is itself a wrapper for cloudpathlib's S3Path, GSPath, and AzureBlobPath). Then update the conditional so that "file", ["s3","s3a"], and the equivalent for Google Cloud Storage and Azure Blob Storage are treated the same and not passed to urljoin.
  2. Add one or more conditionals to handle s3://, gs://, az:// paths specifically.
  3. Reverse the current conditionals so that urls are handled explicitly via if self.__store.fs.protocol in ["http", "https"], and everything else is treated as a file path (using string handling instead of Path which doesn't generalize to cloud paths or adding in cloudpathlib.AnyPath).
will-moore commented 1 year ago

Thanks for the proposed solutions... If options 2 or 3 are sufficient then it would be nicer not to add a dependency on cloudpathlib.

I was just trying to view your data with:

viewer.open("s3://test-bucket-xbahruvc/test.ome.zarr/A/2/0", plugin="napari-ome-zarr")

but even this isn't working for me. I get ValueError: Invalid endpoint: https://s3..amazonaws.com and I don't understand where that is coming from!

Traceback (most recent call last):
  File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 150, in get_json
    data = self.__store.get(subpath)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/_collections_abc.py", line 763, in get
    return self[key]
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/zarr/storage.py", line 1393, in __getitem__
    return self.map[key]
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/mapping.py", line 143, in __getitem__
    result = self.fs.cat(k)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 115, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 100, in sync
    raise return_result
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 55, in _runner
    result[0] = await coro
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 414, in _cat
    raise ex
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
    return await fut
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 1050, in _cat_file
    return await _error_wrapper(_call_and_read, retries=self.retries)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 139, in _error_wrapper
    raise err
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 112, in _error_wrapper
    return await func(*args, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 1037, in _call_and_read
    resp = await self._call_s3(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 340, in _call_s3
    await self.set_session()
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 526, in set_session
    self._s3 = await s3creator.__aenter__()
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/session.py", line 26, in __aenter__
    self._client = await self._coro
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/session.py", line 193, in _create_client
    client = await client_creator.create_client(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/client.py", line 59, in create_client
    client_args = self._get_client_args(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/client.py", line 262, in _get_client_args
    return args_creator.get_client_args(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/args.py", line 70, in get_client_args
    endpoint = endpoint_creator.create_endpoint(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 308, in create_endpoint
    raise ValueError("Invalid endpoint: %s" % endpoint_url)
ValueError: Invalid endpoint: https://s3..amazonaws.com

Tried this on a couple of different conda environments.. Even without napari, I get the same error with:

from ome_zarr import io
zl = io.ZarrLocation("s3://test-bucket-xbahruvc/test.ome.zarr/A/2/0")
elyall commented 1 year ago

Googling the error I get this which says your aws profile is likely not configured correctly. From your error it looks like the path is passed off to s3fs and botocore which is correct, but I'm guessing even when accessing data on a public bucket those packages likely need awscli configured correctly. You can install awscli and run aws configure making sure to pass a valid region (example list here). But you may also need a valid AWS Access Key and AWS Secret Access Key which would require creating an aws account, generating an IAM user, and creating an access key for the user. I'd first try just installing awscli and running aws configure and leave those first two as the default None and then set a valid region (e.g. us-east-1). If that doesn't solve the problem then I can generate a temporary AWS user for you but we'd need to connect via private chat to hand them off as AWS scrapes github looking for leaked credentials.

I did create a solution that doesn't requiring adding a dependency, and I've successfully tested it on the example data above. You can find it at this pull request: https://github.com/ome/ome-zarr-py/pull/322.

joshmoore commented 1 year ago

@will-moore, another option for testing is to create an fsspec config file:

cat ~/.config/fsspec/conf.json
{
    "s3": {
        "anon": true
    }
}
will-moore commented 1 year ago

@joshmoore Thanks, but having created that config.json I'm still seeing the same behaviour :(

$ cat ~/.config/fsspec/conf.json
{
    "s3": {
        "anon": true
    }
}
joshmoore commented 1 year ago

Sorry, which of the behaviors?

will-moore commented 1 year ago

I get the same Error when trying to open with napari as I do with this:

>>> from ome_zarr import io
>>> zr = io.ZarrLocation("s3://test-bucket-xbahruvc/test.ome.zarr/A/2/0")
Error while loading JSON
Traceback (most recent call last):
  File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 150, in get_json
    data = self.__store.get(subpath)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/_collections_abc.py", line 763, in get
    return self[key]
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/zarr/storage.py", line 1393, in __getitem__
    return self.map[key]
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/mapping.py", line 143, in __getitem__
    result = self.fs.cat(k)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 115, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 100, in sync
    raise return_result
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 55, in _runner
    result[0] = await coro
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/fsspec/asyn.py", line 414, in _cat
    raise ex
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
    return await fut
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 1050, in _cat_file
    return await _error_wrapper(_call_and_read, retries=self.retries)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 139, in _error_wrapper
    raise err
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 112, in _error_wrapper
    return await func(*args, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 1037, in _call_and_read
    resp = await self._call_s3(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 340, in _call_s3
    await self.set_session()
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/s3fs/core.py", line 526, in set_session
    self._s3 = await s3creator.__aenter__()
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/session.py", line 26, in __aenter__
    self._client = await self._coro
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/session.py", line 193, in _create_client
    client = await client_creator.create_client(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/client.py", line 59, in create_client
    client_args = self._get_client_args(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/client.py", line 262, in _get_client_args
    return args_creator.get_client_args(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/args.py", line 70, in get_client_args
    endpoint = endpoint_creator.create_endpoint(
  File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 308, in create_endpoint
    raise ValueError("Invalid endpoint: %s" % endpoint_url)
ValueError: Invalid endpoint: https://s3..amazonaws.com
joshmoore commented 1 year ago

Hmmm.... ok. I don't see the problem with or without the config. Either it's due to my platform or some combination of libraries:

conda list ``` $conda list # packages in environment at /opt/homebrew/Caskroom/mambaforge/base/envs/ome-zarr-py: # # Name Version Build Channel aiobotocore 2.7.0 pypi_0 pypi aiohttp 3.8.5 py311heffc1b2_0 conda-forge aioitertools 0.11.0 pyhd8ed1ab_0 conda-forge aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge aom 3.5.0 h7ea286d_0 conda-forge appnope 0.1.3 pyhd8ed1ab_0 conda-forge asciitree 0.3.3 py_2 conda-forge asttokens 2.4.0 pyhd8ed1ab_0 conda-forge async-timeout 4.0.3 pyhd8ed1ab_0 conda-forge attrs 23.1.0 pyh71513ae_1 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge blosc 1.21.5 hc338f07_0 conda-forge botocore 1.31.17 pyhd8ed1ab_3 conda-forge brotli 1.1.0 hb547adb_0 conda-forge brotli-bin 1.1.0 hb547adb_0 conda-forge brotlipy 0.7.0 py311he2be06e_1005 conda-forge brunsli 0.1 h9f76cd9_0 conda-forge bzip2 1.0.8 h3422bc3_4 conda-forge c-blosc2 2.10.2 h068da5f_0 conda-forge ca-certificates 2023.7.22 hf0a4a13_0 conda-forge certifi 2023.7.22 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py311hae827db_3 conda-forge cfgv 3.4.0 pypi_0 pypi charls 2.4.2 h13dd4ca_0 conda-forge charset-normalizer 3.2.0 pyhd8ed1ab_0 conda-forge click 8.1.7 pypi_0 pypi cloudpickle 3.0.0 pypi_0 pypi colorama 0.4.6 pyhd8ed1ab_0 conda-forge cryptography 41.0.3 py311h5fb2c35_0 conda-forge dask 2023.10.1 pypi_0 pypi dav1d 1.2.1 hb547adb_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge distlib 0.3.7 pypi_0 pypi distributed 2023.10.1 pypi_0 pypi entrypoints 0.4 pypi_0 pypi exceptiongroup 1.1.3 pyhd8ed1ab_0 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge fasteners 0.17.3 pyhd8ed1ab_0 conda-forge filelock 3.12.4 pypi_0 pypi flake8 6.1.0 pyhd8ed1ab_0 conda-forge freetype 2.12.1 hadb7bae_2 conda-forge frozenlist 1.4.0 py311heffc1b2_0 conda-forge fsspec 2023.10.0 pypi_0 pypi giflib 5.2.1 h1a8c8d9_3 conda-forge identify 2.5.28 pypi_0 pypi idna 3.4 pyhd8ed1ab_0 conda-forge imagecodecs 2023.9.4 py311hb0b7dac_1 conda-forge imageio 2.31.1 pyh24c5eb1_0 conda-forge importlib-metadata 6.8.0 pypi_0 pypi iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge ipython 8.15.0 pyh31c8845_0 conda-forge jedi 0.19.0 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pypi_0 pypi jmespath 1.0.1 pyhd8ed1ab_0 conda-forge jxrlib 1.1 h27ca646_2 conda-forge lazy_loader 0.3 pyhd8ed1ab_0 conda-forge lcms2 2.15 h40e5a24_2 conda-forge lerc 4.0.0 h9a09cb3_0 conda-forge libaec 1.0.6 hb7217d7_1 conda-forge libavif16 1.0.1 h57fd00a_1 conda-forge libblas 3.9.0 18_osxarm64_openblas conda-forge libbrotlicommon 1.1.0 hb547adb_0 conda-forge libbrotlidec 1.1.0 hb547adb_0 conda-forge libbrotlienc 1.1.0 hb547adb_0 conda-forge libcblas 3.9.0 18_osxarm64_openblas conda-forge libcxx 16.0.6 h4653b0c_0 conda-forge libdeflate 1.18 h1a8c8d9_0 conda-forge libexpat 2.5.0 hb7217d7_1 conda-forge libffi 3.4.2 h3422bc3_5 conda-forge libgfortran 5.0.0 13_2_0_hd922786_1 conda-forge libgfortran5 13.2.0 hf226fd6_1 conda-forge libjpeg-turbo 2.1.5.1 hb547adb_1 conda-forge liblapack 3.9.0 18_osxarm64_openblas conda-forge libopenblas 0.3.24 openmp_hd76b1f2_0 conda-forge libpng 1.6.39 h76d750c_0 conda-forge libsqlite 3.43.0 hb31c410_0 conda-forge libtiff 4.6.0 h23a1a89_0 conda-forge libwebp-base 1.3.2 hb547adb_0 conda-forge libxcb 1.15 hf346824_0 conda-forge libzlib 1.2.13 h53f4e23_5 conda-forge libzopfli 1.0.3 h9f76cd9_0 conda-forge llvm-openmp 16.0.6 h1c12783_0 conda-forge locket 1.0.0 pypi_0 pypi lz4-c 1.9.4 hb7217d7_0 conda-forge markupsafe 2.1.3 pypi_0 pypi matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mccabe 0.7.0 pyhd8ed1ab_0 conda-forge msgpack-python 1.0.5 py311hd6ee22a_0 conda-forge multidict 6.0.4 py311he2be06e_0 conda-forge mypy 1.5.1 py311heffc1b2_0 conda-forge mypy_extensions 1.0.0 pyha770c72_0 conda-forge ncurses 6.4 h7ea286d_0 conda-forge networkx 3.1 pyhd8ed1ab_0 conda-forge nodeenv 1.8.0 pypi_0 pypi numcodecs 0.11.0 py311ha397e9f_2 conda-forge numpy 1.25.2 py311hb8f3215_0 conda-forge ome-zarr 0.8.2 pypi_0 pypi openjpeg 2.5.0 h4c1507b_3 conda-forge openssl 3.1.2 h53f4e23_0 conda-forge packaging 23.1 pyhd8ed1ab_0 conda-forge pandas 2.1.0 py311h9e438b8_0 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge partd 1.4.1 pypi_0 pypi pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 10.0.0 py311he9c13d2_1 conda-forge pip 23.2.1 pyhd8ed1ab_0 conda-forge platformdirs 3.10.0 pypi_0 pypi pluggy 1.3.0 pyhd8ed1ab_0 conda-forge pre-commit 3.4.0 pypi_0 pypi prompt-toolkit 3.0.39 pyha770c72_0 conda-forge prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge psutil 5.9.5 py311he2be06e_0 conda-forge pthread-stubs 0.4 h27ca646_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge pycodestyle 2.11.0 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pyflakes 3.1.0 pyhd8ed1ab_0 conda-forge pygments 2.16.1 pyhd8ed1ab_0 conda-forge pyopenssl 23.2.0 pyhd8ed1ab_1 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge pytest 7.4.2 pyhd8ed1ab_0 conda-forge python 3.11.5 h47c9636_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge python_abi 3.11 3_cp311 conda-forge pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge pywavelets 1.4.1 py311h4add359_0 conda-forge pyyaml 6.0.1 pypi_0 pypi rav1e 0.6.6 h69fbcac_2 conda-forge readline 8.2 h92ec313_1 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge s3fs 2023.10.0 pypi_0 pypi scikit-image 0.21.0 py311ha891d26_0 conda-forge scipy 1.11.2 py311h93d07a4_1 conda-forge setuptools 68.2.2 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.10 h17c5cce_0 conda-forge sortedcontainers 2.4.0 pypi_0 pypi stack_data 0.6.2 pyhd8ed1ab_0 conda-forge svt-av1 1.7.0 hb765f3a_0 conda-forge tblib 3.0.0 pypi_0 pypi tifffile 2023.8.30 pyhd8ed1ab_0 conda-forge tk 8.6.12 he1e0b03_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge toolz 0.12.0 pypi_0 pypi tornado 6.3.3 pypi_0 pypi traitlets 5.10.0 pyhd8ed1ab_0 conda-forge typing-extensions 4.7.1 hd8ed1ab_0 conda-forge typing_extensions 4.7.1 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge urllib3 1.26.15 pyhd8ed1ab_0 conda-forge virtualenv 20.24.5 pypi_0 pypi wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge wheel 0.41.2 pyhd8ed1ab_0 conda-forge wrapt 1.15.0 py311he2be06e_0 conda-forge xarray 2023.8.0 pyhd8ed1ab_0 conda-forge xorg-libxau 1.0.11 hb547adb_0 conda-forge xorg-libxdmcp 1.1.3 h27ca646_0 conda-forge xz 5.2.6 h57fd34a_0 conda-forge yarl 1.9.2 py311heffc1b2_0 conda-forge zarr 2.16.1 pyhd8ed1ab_0 conda-forge zfp 1.0.0 hb6e4faa_3 conda-forge zict 3.0.0 pypi_0 pypi zipp 3.17.0 pypi_0 pypi zlib-ng 2.0.7 h1a8c8d9_0 conda-forge zstd 1.5.5 h4f39d0f_0 conda-forge ```
will-moore commented 1 year ago

Hmm - I seem to generally have slightly older packages than you (and a load of omero-web stuff).

conda list ``` $ conda list # packages in environment at /Users/wmoore/opt/anaconda3/envs/omeroweb2: # # Name Version Build Channel aiobotocore 2.4.2 pypi_0 pypi aiohttp 3.8.4 pypi_0 pypi aioitertools 0.11.0 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi alabaster 0.7.13 pypi_0 pypi anndata 0.9.0rc1 pypi_0 pypi app-model 0.1.2 pypi_0 pypi appdirs 1.4.4 pyh9f0ad1d_0 conda-forge appnope 0.1.3 pypi_0 pypi asciitree 0.3.3 pypi_0 pypi asgiref 3.6.0 pypi_0 pypi asttokens 2.2.1 pypi_0 pypi async-timeout 4.0.2 pypi_0 pypi attrs 22.2.0 pypi_0 pypi babel 2.12.1 pypi_0 pypi backcall 0.2.0 pypi_0 pypi black 23.3.0 pypi_0 pypi blosc2 2.0.0 pypi_0 pypi botocore 1.27.59 pypi_0 pypi brotlipy 0.7.0 py39ha30fb19_1005 conda-forge build 0.10.0 pypi_0 pypi bump2version 1.0.1 pypi_0 pypi bumpversion 0.6.0 pypi_0 pypi bzip2 1.0.8 h0d85af4_4 conda-forge ca-certificates 2022.12.7 h033912b_0 conda-forge cachetools 5.3.0 pypi_0 pypi cachey 0.2.1 pypi_0 pypi certifi 2022.12.7 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py39h131948b_3 conda-forge chardet 5.1.0 pypi_0 pypi charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge check-jsonschema 0.22.0 pypi_0 pypi click 8.1.3 pypi_0 pypi cloudpickle 2.2.1 pypi_0 pypi colorama 0.4.6 pypi_0 pypi comm 0.1.2 pypi_0 pypi concurrent-log-handler 0.9.20 pypi_0 pypi contourpy 1.0.7 pypi_0 pypi cryptography 39.0.0 py39h7eb6a14_0 conda-forge cycler 0.11.0 pypi_0 pypi cython 3.0.0 pypi_0 pypi dask 2023.5.0 pypi_0 pypi dask-image 2022.9.0 pypi_0 pypi debugpy 1.6.6 pypi_0 pypi decorator 5.1.1 pypi_0 pypi distlib 0.3.6 pypi_0 pypi distributed 2023.5.0 pypi_0 pypi django 4.2.6 pypi_0 pypi django-cors-headers 3.7.0 pypi_0 pypi django-pipeline 2.1.0 pypi_0 pypi django-redis 5.0.0 pypi_0 pypi docrep 0.3.2 pypi_0 pypi docstring-parser 0.15 pypi_0 pypi docutils 0.17.1 pypi_0 pypi entrypoints 0.4 pypi_0 pypi exceptiongroup 1.1.1 pypi_0 pypi executing 1.2.0 pypi_0 pypi ezomero 1.2.1 pypi_0 pypi fasteners 0.18 pypi_0 pypi filelock 3.12.0 pypi_0 pypi flake8 6.0.0 pypi_0 pypi flask 2.2.2 pypi_0 pypi fonttools 4.39.0 pypi_0 pypi freetype 2.12.1 h3f81eb7_1 conda-forge freetype-py 2.3.0 pypi_0 pypi frozenlist 1.3.3 pypi_0 pypi fsspec 2023.3.0 pypi_0 pypi future 0.18.3 pyhd8ed1ab_0 conda-forge gunicorn 20.1.0 pypi_0 pypi h5py 3.8.0 pypi_0 pypi heapdict 1.0.1 pypi_0 pypi hsluv 5.0.3 pypi_0 pypi idna 3.4 pyhd8ed1ab_0 conda-forge idr-gallery 3.9.3.dev0 dev_0 igraph 0.10.4 pypi_0 pypi imageio 2.26.0 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi importlib-metadata 6.0.0 pypi_0 pypi importlib-resources 5.12.0 pypi_0 pypi in-n-out 0.1.7 pypi_0 pypi inflect 6.0.2 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi iohub 0.1.0.dev3 pypi_0 pypi ipykernel 6.21.3 pypi_0 pypi ipython 8.11.0 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi itsdangerous 2.1.2 pypi_0 pypi jedi 0.18.2 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi jmespath 1.0.1 pypi_0 pypi joblib 1.2.0 pypi_0 pypi jsonschema 4.17.3 pypi_0 pypi jupyter-client 8.0.3 pypi_0 pypi jupyter-core 5.2.0 pypi_0 pypi kiwisolver 1.4.4 pypi_0 pypi lazy-loader 0.1 pypi_0 pypi lcms2 2.15 h2dcdeff_1 conda-forge leidenalg 0.9.1 pypi_0 pypi lerc 4.0.0 hb486fe8_0 conda-forge libblas 3.9.0 16_osx64_openblas conda-forge libcblas 3.9.0 16_osx64_openblas conda-forge libcst 0.4.9 pypi_0 pypi libcxx 15.0.7 h71dddab_0 conda-forge libdeflate 1.17 hac1461d_0 conda-forge libffi 3.4.2 h0d85af4_5 conda-forge libgfortran 5.0.0 11_3_0_h97931a8_31 conda-forge libgfortran5 12.2.0 he409387_31 conda-forge libiconv 1.17 hac89ed1_0 conda-forge libjpeg-turbo 2.1.5.1 hb7f2c08_0 conda-forge liblapack 3.9.0 16_osx64_openblas conda-forge libopenblas 0.3.21 openmp_h429af6e_3 conda-forge libpng 1.6.39 ha978bb4_0 conda-forge libsqlite 3.40.0 ha978bb4_0 conda-forge libtiff 4.5.0 hd920806_5 conda-forge libwebp-base 1.3.0 hb7f2c08_0 conda-forge libxcb 1.13 h0d85af4_1004 conda-forge libzlib 1.2.13 hfd90126_4 conda-forge llvm-openmp 15.0.7 h61d9ccf_0 conda-forge llvmlite 0.38.1 pypi_0 pypi locket 1.0.0 pypi_0 pypi magicgui 0.7.2 pypi_0 pypi markdown-it-py 2.2.0 pypi_0 pypi markupsafe 2.1.2 pypi_0 pypi matplotlib 3.7.1 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi matplotlib-scalebar 0.8.1 pypi_0 pypi mccabe 0.7.0 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mdv 0.0.1 pypi_0 pypi minimal-webapp 0.0.1 pypi_0 pypi msgpack 1.0.5 pypi_0 pypi multidict 6.0.4 pypi_0 pypi mypy-extensions 1.0.0 pypi_0 pypi napari 0.4.17 pypi_0 pypi napari-console 0.0.7 pypi_0 pypi napari-ome-zarr 0.5.2 pypi_0 pypi napari-omero 0.2.1.dev5+gd97a662 pypi_0 pypi napari-plugin-engine 0.2.0 pypi_0 pypi napari-svg 0.1.6 pypi_0 pypi natsort 8.3.1 pypi_0 pypi ncurses 6.3 h96cf925_1 conda-forge ndtiff 1.12.1 pypi_0 pypi nest-asyncio 1.5.6 pypi_0 pypi networkx 3.0 pypi_0 pypi ngff-tables-prototype 0.1.dev42+g0821bbb.d20230314 dev_0 no-implicit-optional 1.3 pypi_0 pypi npe2 0.6.2 pypi_0 pypi numba 0.55.2 pypi_0 pypi numcodecs 0.11.0 pypi_0 pypi numexpr 2.8.5 pypi_0 pypi numpy 1.24.2 py39h6ee2318_0 conda-forge numpydoc 1.5.0 pypi_0 pypi ome-zarr 0.7.2.dev0 dev_0 omero-cli-render 0.8.1.dev0 dev_0 omero-cli-zarr 0.5.2.dev44+g7baf223 dev_0 omero-demo-cleanup 0.1.1.dev0 dev_0 omero-figure 5.1.1.dev0 dev_0 omero-gallery 3.4.3.dev0 dev_0 omero-iviewer 0.14.0.dev0 dev_0 omero-mapr 0.5.2.dev0 dev_0 omero-marshal 0.8.0 pypi_0 pypi omero-mdv 0.0.1 pypi_0 pypi omero-mkngff 0.1.0.dev0 dev_0 omero-parade 0.2.4.dev0 dev_0 omero-py 5.13.2.dev0 dev_0 omero-rois 0.3.0 pypi_0 pypi omero-search-engine-client 0.0.1.dev0 dev_0 omero-tagsearch 4.0.0 pypi_0 pypi omero-web 5.22.2.dev0 dev_0 omero-web-zarr 0.0.2 dev_0 omero-webtagging-autotag 3.2.0 pypi_0 pypi omero-webtagging-tagsearch 3.2.0 pypi_0 pypi omero2pandas 0.1.2 pypi_0 pypi omnipath 1.0.6 pypi_0 pypi openjpeg 2.5.0 h13ac156_2 conda-forge openssl 1.1.1t hfd90126_0 conda-forge packaging 23.1 pypi_0 pypi pandas 1.5.3 pypi_0 pypi parade-crossfilter 0.0.6 dev_0 parso 0.8.3 pypi_0 pypi partd 1.3.0 pypi_0 pypi pathspec 0.11.1 pypi_0 pypi patsy 0.5.3 pypi_0 pypi pexpect 4.8.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 9.4.0 py39h77c96bc_2 conda-forge pims 0.6.1 pypi_0 pypi pint 0.20.1 pypi_0 pypi pip 23.0.1 pyhd8ed1ab_0 conda-forge platformdirs 3.5.0 pypi_0 pypi pluggy 1.0.0 pypi_0 pypi pooch 1.7.0 pypi_0 pypi portalocker 2.7.0 pypi_0 pypi prompt-toolkit 3.0.38 pypi_0 pypi psutil 5.9.4 pypi_0 pypi psygnal 0.8.1 pypi_0 pypi pthread-stubs 0.4 hc929b4f_1001 conda-forge ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.2 pypi_0 pypi py-cpuinfo 9.0.0 pypi_0 pypi pycodestyle 2.10.0 pypi_0 pypi pycparser 2.21 pyhd8ed1ab_0 conda-forge pydantic 1.10.6 pypi_0 pypi pydantic-ome-ngff 0.2.2 pypi_0 pypi pyflakes 3.0.1 pypi_0 pypi pygments 2.14.0 pypi_0 pypi pynndescent 0.5.8 pypi_0 pypi pyopengl 3.1.6 pypi_0 pypi pyopenssl 23.0.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pypi_0 pypi pyproject-api 1.5.1 pypi_0 pypi pyproject-hooks 1.0.0 pypi_0 pypi pyqt5 5.15.9 pypi_0 pypi pyqt5-qt5 5.15.2 pypi_0 pypi pyqt5-sip 12.11.1 pypi_0 pypi pyrsistent 0.19.3 pypi_0 pypi pysocks 1.7.1 pyha2e5f31_6 conda-forge pytest 7.3.1 pypi_0 pypi python 3.9.15 h531fd05_0_cpython conda-forge python-dateutil 2.8.2 pypi_0 pypi python_abi 3.9 3_cp39 conda-forge pytomlpp 1.0.11 pypi_0 pypi pytz 2022.7.1 pypi_0 pypi pywavelets 1.4.1 pypi_0 pypi pywin32-on-windows 0.1.0 pyh1179c8e_3 conda-forge pyyaml 6.0 py39ha30fb19_5 conda-forge pyzmq 25.0.1 pypi_0 pypi qtconsole 5.4.1 pypi_0 pypi qtpy 2.3.0 pypi_0 pypi readline 8.1.2 h3899abd_0 conda-forge redis 4.5.2 pypi_0 pypi requests 2.28.2 pyhd8ed1ab_0 conda-forge rich 13.3.2 pypi_0 pypi ruamel-yaml 0.17.21 pypi_0 pypi ruamel-yaml-clib 0.2.7 pypi_0 pypi s3fs 2023.3.0 pypi_0 pypi scanpy 1.9.3 pypi_0 pypi scikit-image 0.20.0 pypi_0 pypi scikit-learn 1.2.2 pypi_0 pypi scipy 1.9.1 pypi_0 pypi seaborn 0.12.2 pypi_0 pypi session-info 1.0.0 pypi_0 pypi setuptools 67.6.0 pyhd8ed1ab_0 conda-forge six 1.16.0 pypi_0 pypi slicerator 1.1.0 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi sortedcontainers 2.4.0 pypi_0 pypi sphinx 4.5.0 pypi_0 pypi sphinxcontrib-applehelp 1.0.4 pypi_0 pypi sphinxcontrib-devhelp 1.0.2 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.1 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.3 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.5 pypi_0 pypi sqlparse 0.4.3 pypi_0 pypi squidpy 1.2.3 pypi_0 pypi stack-data 0.6.2 pypi_0 pypi statsmodels 0.13.5 pypi_0 pypi stdlib-list 0.8.0 pypi_0 pypi superqt 0.4.1 pypi_0 pypi tables 3.8.0 pypi_0 pypi tblib 1.7.0 pypi_0 pypi texttable 1.6.7 pypi_0 pypi threadpoolctl 3.1.0 pypi_0 pypi tifffile 2023.2.28 pypi_0 pypi tk 8.6.12 h5dbffcc_0 conda-forge tomli 2.0.1 pypi_0 pypi toolz 0.12.0 pypi_0 pypi tornado 6.2 pypi_0 pypi tox 4.5.1 pypi_0 pypi tqdm 4.65.0 pypi_0 pypi traitlets 5.9.0 pypi_0 pypi typer 0.7.0 pypi_0 pypi typing-extensions 4.5.0 pypi_0 pypi typing-inspect 0.8.0 pypi_0 pypi tzdata 2022g h191b570_0 conda-forge umap-learn 0.5.3 pypi_0 pypi urllib3 1.26.15 pyhd8ed1ab_0 conda-forge validators 0.20.0 pypi_0 pypi virtualenv 20.23.0 pypi_0 pypi vispy 0.11.0 pypi_0 pypi wcwidth 0.2.6 pypi_0 pypi werkzeug 2.2.2 pypi_0 pypi wheel 0.38.4 pyhd8ed1ab_0 conda-forge whitenoise 6.4.0 pypi_0 pypi wrapt 1.15.0 pypi_0 pypi xarray 2023.2.0 pypi_0 pypi xorg-libxau 1.0.9 h35c211d_0 conda-forge xorg-libxdmcp 1.1.3 h35c211d_0 conda-forge xz 5.2.6 h775f41a_0 conda-forge yaml 0.2.5 h0d85af4_2 conda-forge yarl 1.8.2 pypi_0 pypi zarr 2.14.2 pypi_0 pypi zeroc-ice 3.6.5 py39h41db984_4 conda-forge zict 3.0.0 pypi_0 pypi zipp 3.15.0 pypi_0 pypi zstd 1.5.2 hbc0c0cd_6 conda-forge ```

Tried with a fresh conda env...

conda create -n omezarr python=3.9
conda activate omezarr
pip install ome-zarr
python
Python 3.9.18 (main, Sep 11 2023, 08:38:23) 
[Clang 14.0.6 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ome_zarr import io
>>> zr = io.ZarrLocation("s3://test-bucket-xbahruvc/test.ome.zarr/A/2/0")
same error ``` Error while loading JSON Traceback (most recent call last): File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/ome_zarr/io.py", line 150, in get_json data = self.__store.get(subpath) File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/_collections_abc.py", line 763, in get return self[key] File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/zarr/storage.py", line 1428, in __getitem__ return self.map[key] File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/fsspec/mapping.py", line 151, in __getitem__ result = self.fs.cat(k) File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/fsspec/asyn.py", line 118, in wrapper return sync(self.loop, func, *args, **kwargs) File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/fsspec/asyn.py", line 103, in sync raise return_result File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/fsspec/asyn.py", line 56, in _runner result[0] = await coro File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/fsspec/asyn.py", line 446, in _cat raise ex File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for return await fut File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/s3fs/core.py", line 1102, in _cat_file return await _error_wrapper(_call_and_read, retries=self.retries) File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/s3fs/core.py", line 140, in _error_wrapper raise err File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/s3fs/core.py", line 113, in _error_wrapper return await func(*args, **kwargs) File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/s3fs/core.py", line 1089, in _call_and_read resp = await self._call_s3( File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/s3fs/core.py", line 341, in _call_s3 await self.set_session() File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/s3fs/core.py", line 527, in set_session self._s3 = await s3creator.__aenter__() File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/aiobotocore/session.py", line 27, in __aenter__ self._client = await self._coro File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/aiobotocore/session.py", line 211, in _create_client client = await client_creator.create_client( File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/aiobotocore/client.py", line 76, in create_client client_args = self._get_client_args( File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/aiobotocore/client.py", line 267, in _get_client_args return args_creator.get_client_args( File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/aiobotocore/args.py", line 76, in get_client_args endpoint = endpoint_creator.create_endpoint( File "/Users/wmoore/opt/anaconda3/envs/omezarr/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 308, in create_endpoint raise ValueError("Invalid endpoint: %s" % endpoint_url) ValueError: Invalid endpoint: https://s3..amazonaws.com ```
conda list ``` conda list # packages in environment at /Users/wmoore/opt/anaconda3/envs/omezarr: # # Name Version Build Channel aiobotocore 2.7.0 pypi_0 pypi aiohttp 3.8.6 pypi_0 pypi aioitertools 0.11.0 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi asciitree 0.3.3 pypi_0 pypi async-timeout 4.0.3 pypi_0 pypi attrs 23.1.0 pypi_0 pypi botocore 1.31.64 pypi_0 pypi ca-certificates 2023.08.22 hecd8cb5_0 certifi 2023.7.22 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi click 8.1.7 pypi_0 pypi cloudpickle 3.0.0 pypi_0 pypi dask 2023.10.1 pypi_0 pypi distributed 2023.10.1 pypi_0 pypi fasteners 0.19 pypi_0 pypi frozenlist 1.4.0 pypi_0 pypi fsspec 2023.10.0 pypi_0 pypi idna 3.4 pypi_0 pypi imageio 2.31.6 pypi_0 pypi importlib-metadata 6.8.0 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi jmespath 1.0.1 pypi_0 pypi lazy-loader 0.3 pypi_0 pypi libcxx 14.0.6 h9765a3e_0 libffi 3.4.4 hecd8cb5_0 locket 1.0.0 pypi_0 pypi markupsafe 2.1.3 pypi_0 pypi msgpack 1.0.7 pypi_0 pypi multidict 6.0.4 pypi_0 pypi ncurses 6.4 hcec6c5f_0 networkx 3.2.1 pypi_0 pypi numcodecs 0.12.1 pypi_0 pypi numpy 1.26.1 pypi_0 pypi ome-zarr 0.8.2 pypi_0 pypi openssl 3.0.11 hca72f7f_2 packaging 23.2 pypi_0 pypi partd 1.4.1 pypi_0 pypi pillow 10.0.1 pypi_0 pypi pip 23.3 py39hecd8cb5_0 psutil 5.9.6 pypi_0 pypi python 3.9.18 h5ee71fb_0 python-dateutil 2.8.2 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi readline 8.2 hca72f7f_0 requests 2.31.0 pypi_0 pypi s3fs 2023.10.0 pypi_0 pypi scikit-image 0.22.0 pypi_0 pypi scipy 1.11.3 pypi_0 pypi setuptools 68.0.0 py39hecd8cb5_0 six 1.16.0 pypi_0 pypi sortedcontainers 2.4.0 pypi_0 pypi sqlite 3.41.2 h6c40b1e_0 tblib 3.0.0 pypi_0 pypi tifffile 2023.9.26 pypi_0 pypi tk 8.6.12 h5d9f67b_0 toolz 0.12.0 pypi_0 pypi tornado 6.3.3 pypi_0 pypi typing-extensions 4.8.0 pypi_0 pypi tzdata 2023c h04d1e81_0 urllib3 1.26.18 pypi_0 pypi wheel 0.41.2 py39hecd8cb5_0 wrapt 1.15.0 pypi_0 pypi xz 5.4.2 h6c40b1e_0 yarl 1.9.2 pypi_0 pypi zarr 2.16.1 pypi_0 pypi zict 3.0.0 pypi_0 pypi zipp 3.17.0 pypi_0 pypi zlib 1.2.13 h4dc903c_0 ```
elyall commented 1 year ago

This might be relevant, the test file is in a bucket in AWS region us-west-2.

elyall commented 12 months ago

@will-moore ValueError: Invalid endpoint: https://s3..amazonaws.com does show that somehow you're not incorporating the AWS region correctly as a valid endpoint would be something like https://s3.us-west-2.amazonaws.com where the region is incorporated.

I would like to get the pull request incorporated and keep it from going stale. How do we do that? I would also like to take that test file down.

will-moore commented 12 months ago

Thanks for that info... I tried updating my fsspec config like this, and it worked!

$ cat ~/.config/fsspec/conf.json
{
    "s3": {
        "anon": true,
        "endpoint_url": "https://s3.us-west-2.amazonaws.com"
    }
}

With that, and your PR #322, that fixed the Could not find first well Exception and I was able to view the single-Well Plate

napari --plugin napari-ome-zarr "s3://test-bucket-xbahruvc/test.ome.zarr"

Screenshot 2023-11-13 at 16 23 41