ome / omero-cli-zarr

https://pypi.org/project/omero-cli-zarr/
GNU General Public License v2.0
15 stars 10 forks source link

Remove np.bool alias #143

Closed will-moore closed 1 year ago

will-moore commented 1 year ago

As reported by @jburel, trying to export labels with recent numpy (e.g. numpy==1.24.3) fails (Error below).

See https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

Error ``` $ omero zarr masks Image:6001240 /Users/wmoore/Desktop/ZARR/omero-cli-zarr/src/omero_zarr/masks.py:54: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar. 1: np.bool, Error loading: /Users/wmoore/Desktop/ZARR/omero-cli-zarr/src/omero/plugins/zarr.py Traceback (most recent call last): File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/omero/cli.py", line 1690, in loadpath execfile(str(pathobj), loc) File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/past/builtins/misc.py", line 154, in execfile exec_(code, myglobals, mylocals) File "/Users/wmoore/Desktop/ZARR/omero-cli-zarr/src/omero/plugins/zarr.py", line 1, in from omero_zarr.cli import HELP, ZarrControl File "/Users/wmoore/Desktop/ZARR/omero-cli-zarr/src/omero_zarr/cli.py", line 32, in from .masks import ( File "/Users/wmoore/Desktop/ZARR/omero-cli-zarr/src/omero_zarr/masks.py", line 54, in 1: np.bool, File "/Users/wmoore/opt/anaconda3/envs/omeroweb2/lib/python3.9/site-packages/numpy/__init__.py", line 305, in __getattr__ raise AttributeError(__former_attrs__[attr]) AttributeError: module 'numpy' has no attribute 'bool'. `np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations ```

This fixes as described.

To test:

$ omero login   # to IDR
$ omero zarr export Image:6001240
$ omero zarr masks Image:6001240
will-moore commented 1 year ago

Merge and release as 0.5.2?

pwalczysko commented 1 year ago

Sorry, I am having an error on freshly created env with omero-py on Python 3.9 and this branch

(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ git branch
* (HEAD detached at will/numpy_aliases_fix)
  master
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ conda list | grep numpy
numpy                     1.24.3           py39h892e69a_0    conda-forge
omero zarr export Image:6001240
....
... # successful execution, no errors, but see below
omero zarr masks Image:6001240
Using session for public@idr.openmicroscopy.org:4064. Idle timeout: 10 min. Current group: Public
Export Masks on Image: B1_C1.tif
Found 2367 mask shapes in 61 ROIs
Unique dimensions: {'T': {None}, 'C': {None}, 'Z': {21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235}}
source_image 6001240.zarr
Ignoring dimensions {'C', 'T'}
Traceback (most recent call last):
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/bin/omero", line 10, in <module>
    sys.exit(main())
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/omero/main.py", line 126, in main
    rv = omero.cli.argv()
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/omero/cli.py", line 1787, in argv
    cli.invoke(args[1:])
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/omero/cli.py", line 1225, in invoke
    stop = self.onecmd(line, previous_args)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/omero/cli.py", line 1302, in onecmd
    self.execute(line, previous_args)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/omero/cli.py", line 1384, in execute
    args.func(args)
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/cli.py", line 125, in _wrapper
    return func(self, *args, **kwargs)
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/cli.py", line 309, in masks
    image_shapes_to_zarr(image, ["Mask"], args)
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/masks.py", line 199, in image_shapes_to_zarr
    saver.save(list(masks.values()), args.label_name)
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/masks.py", line 372, in save
    write_multiscale_labels(
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/ome_zarr/writer.py", line 727, in write_multiscale_labels
    dask_delayed_jobs = write_multiscale(
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/ome_zarr/writer.py", line 266, in write_multiscale
    group.create_dataset(str(path), data=data, chunks=chunks_opt, **options)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/hierarchy.py", line 1043, in create_dataset
    return self._write_op(self._create_dataset_nosync, name, **kwargs)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/hierarchy.py", line 895, in _write_op
    return f(*args, **kwargs)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/hierarchy.py", line 1060, in _create_dataset_nosync
    a = array(data, store=self._store, path=path, chunk_store=self._chunk_store,
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/creation.py", line 394, in array
    z = create(**kwargs)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/creation.py", line 178, in create
    init_array(store, shape=shape, chunks=chunks, dtype=dtype, compressor=compressor,
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/storage.py", line 437, in init_array
    _init_array_metadata(store, shape=shape, chunks=chunks, dtype=dtype,
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/zarr/storage.py", line 498, in _init_array_metadata
    raise ContainsArrayError(path)
zarr.errors.ContainsArrayError: path 'labels/0/0' contains an array
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ python --version
Python 3.9.15
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ pip freeze | grep numpy
numpy @ file:///Users/runner/miniforge3/conda-bld/numpy_1682210351842/work
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ pip install numpy==1.24.3
Requirement already satisfied: numpy==1.24.3 in /Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages (1.24.3)
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ git branch
* (HEAD detached at will/numpy_aliases_fix)
  master
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ conda list | grep numpy
numpy                     1.24.3           py39h892e69a_0    conda-forge
pwalczysko commented 1 year ago

When I checkout master, I have the error which this PR is trying to fix

(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ git checkout master
Previous HEAD position was e78a891 Remove np.bool alias
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
(myenv) pwalczysko@ls31618~/Work/omero-cli-zarr$ omero zarr masks Image:6001240
/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/masks.py:54: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  1: np.bool,
Error loading: /Users/pwalczysko/Work/omero-cli-zarr/src/omero/plugins/zarr.py
Traceback (most recent call last):
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/omero/cli.py", line 1693, in loadpath
    execfile(str(pathobj), loc)
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/past/builtins/misc.py", line 154, in execfile
    exec_(code, myglobals, mylocals)
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero/plugins/zarr.py", line 1, in <module>
    from omero_zarr.cli import HELP, ZarrControl
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/cli.py", line 32, in <module>
    from .masks import (
  File "/Users/pwalczysko/Work/omero-cli-zarr/src/omero_zarr/masks.py", line 54, in <module>
    1: np.bool,
  File "/Users/pwalczysko/opt/anaconda3/envs/myenv/lib/python3.9/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
usage: /Users/pwalczysko/opt/anaconda3/envs/myenv/bin/omero
       [-h] [-v] [-d DEBUG] [--path PATH] [-C] [-s SERVER] [-p PORT]
       [-g GROUP] [-u USER] [-w PASSWORD] [-k KEY] [--sudo ADMINUSER] [-q]
       <subcommand> ...
/Users/pwalczysko/opt/anaconda3/envs/myenv/bin/omero: error: argument <subcommand>: invalid choice: 'zarr'

choose from:
    admin, chgrp, chown, config, db, delete, download, errors, export, 
    fs, group, help, hql, import, ldap, load, login, logout, node, 
    obj, perf, quit, script, search, sessions, shell, tag, testengine, 
    upload, user, version
pwalczysko commented 1 year ago

In summary, I think this PR is fixing the error with np.bool. But the test is not passing because of another error,

zarr.errors.ContainsArrayError: path 'labels/0/0' contains an array

jburel commented 1 year ago

@pwalczysko could you try with 6001247 since it is the one we will be using

pwalczysko commented 1 year ago

@pwalczysko could you try with 6001247 since it is the one we will be using

Yes, 6001247 went fine.

omero zarr masks Image:6001247
Using session for public@idr.openmicroscopy.org:4064. Idle timeout: 10 min. Current group: Public
Export Masks on Image: B4_C3.tif
Found 1771 mask shapes in 48 ROIs
Unique dimensions: {'T': {None}, 'C': {None}, 'Z': {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256}}
source_image 6001247.zarr
Ignoring dimensions {'C', 'T'}
jburel commented 1 year ago

Great Thanks @will-moore could you merge and tag please?

will-moore commented 1 year ago

Released 0.5.2 👍

jburel commented 1 year ago

thanks