ValueError when running BlobDetector

iimog commented 1 year ago

Description

When trying to run the BlobDetector example, I receive a "ValueError: Data must be 1-dimensional"

Steps/Code to Reproduce

To reproduce, install the latest version of starfish from GitHub (0.2.2-39-g853f56c7) and try to run the BlobDetector example. Code from that example copied here:

# Load in situ sequencing experiment
from starfish.image import ApplyTransform, LearnTransform, Filter
from starfish.types import Axes
from starfish import data, FieldOfView
from starfish.spots import FindSpots
from starfish.util.plot import imshow_plane
experiment = data.ISS()
fov = experiment.fov()
imgs = fov.get_image(FieldOfView.PRIMARY_IMAGES) # primary images
dots = fov.get_image("dots") # reference round where every spot labeled with fluorophore

# filter raw data
masking_radius = 15
filt = Filter.WhiteTophat(masking_radius, is_volume=False)
filt.run(imgs, in_place=True)
filt.run(dots, in_place=True)

# register primary images to reference round
learn_translation = LearnTransform.Translation(reference_stack=dots, axes=Axes.ROUND, upsampling=1000)
transforms_list = learn_translation.run(imgs.reduce({Axes.CH, Axes.ZPLANE}, func="max"))
warp = ApplyTransform.Warp()
warp.run(imgs, transforms_list=transforms_list, in_place=True)

# view dots to estimate radius of spots: radius range from 1.5 to 4 pixels
imshow_plane(dots, {Axes.X: (500, 550), Axes.Y: (600, 650)})

# run blob detector with dots as reference image
# following guideline of sigma = radius/sqrt(2) for 2D images
# threshold is set conservatively low
bd = FindSpots.BlobDetector(
    min_sigma=1,
    max_sigma=3,
    num_sigma=10,
    threshold=0.01,
    is_volume=False,
    measurement_type='mean',
)
spots = bd.run(image_stack=imgs, reference_image=dots)

Expected Results

No error is thrown.

Actual Results

The following error occurs:

100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 112.34it/s]
100%|███████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 69.52it/s]
100%|██████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 270.05it/s]
Traceback (most recent call last):
  File "/home/markus/Downloads/blob_detector.py", line 98, in <module>
    spots = bd.run(image_stack=imgs, reference_image=dots)
  File "/home/markus/software/starfish/starfish/core/pipeline/algorithmbase.py", line 23, in helper
    result = func(*args, **kwargs)
  File "/home/markus/software/starfish/starfish/core/spots/FindSpots/blob.py", line 196, in run
    reference_spots = spot_finding_method(data_image)
  File "/home/markus/software/starfish/starfish/core/spots/FindSpots/blob.py", line 160, in image_to_spots
    Features.SPOT_RADIUS: radius,
  File "/home/markus/miniconda3/envs/starfish/lib/python3.7/site-packages/pandas/core/frame.py", line 614, in __init__
    mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
  File "/home/markus/miniconda3/envs/starfish/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 465, in dict_to_mgr
    arrays, data_names, index, columns, dtype=dtype, typ=typ, consolidate=copy
  File "/home/markus/miniconda3/envs/starfish/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 124, in arrays_to_mgr
    arrays = _homogenize(arrays, index, dtype)
  File "/home/markus/miniconda3/envs/starfish/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 590, in _homogenize
    val, index, dtype=dtype, copy=False, raise_cast_failure=False
  File "/home/markus/miniconda3/envs/starfish/lib/python3.7/site-packages/pandas/core/construction.py", line 577, in sanitize_array
    subarr = _sanitize_ndim(subarr, data, dtype, index, allow_2d=allow_2d)
  File "/home/markus/miniconda3/envs/starfish/lib/python3.7/site-packages/pandas/core/construction.py", line 628, in _sanitize_ndim
    raise ValueError("Data must be 1-dimensional")
ValueError: Data must be 1-dimensional

Versions

Linux-5.19.0-35-generic-x86_64-with-debian-bookworm-sid Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21) [GCC 9.4.0] NumPy 1.21.2 SciPy 1.7.1 scikit-image 0.18.3 pandas 1.3.2 sklearn 0.24.2 xarray 0.19.0 sympy 1.5.1 starfish 0.2.2+39.g853f56c7

sebgoti commented 8 months ago

I have not tried to figure out the exact source of the problem but it could be something breaking from an incompatible dependency. Try installing this: $ pip install starfish==0.2.2 $ pip install scikit-image==0.15.0

and then you should be able to run the blob detector as above.

iimog commented 8 months ago

Thank you for your response. I'm quite sure, that you are correct, and the error is due to incompatible dependencies. However, I still don't get a working environment with an installation like this. What I tried:

mamba create -n starfish-issue-test python=3.8
mamba activate starfish-issue-test
pip install starfish==0.2.2
pip install scikit-image==0.15.0

Running the example above now leads to the new error (truncated):

[...]
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

Installing an older version of numpy (I tried multiple, the following message is with version 1.20.3) leads to

Traceback (most recent call last):
  File "example.py", line 2, in <module>
    from starfish.image import ApplyTransform, LearnTransform, Filter
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/starfish/__init__.py", line 1, in <module>
    from . import (
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/starfish/image.py", line 1, in <module>
    from starfish.core.image import (  # noqa: F401
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/starfish/core/image/__init__.py", line 1, in <module>
    from ._registration import ApplyTransform
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/starfish/core/image/_registration/ApplyTransform/__init__.py", line 1, in <module>
    from ._base import ApplyTransformAlgorithm
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/starfish/core/image/_registration/ApplyTransform/_base.py", line 4, in <module>
    from starfish.core.imagestack.imagestack import ImageStack
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/starfish/core/imagestack/imagestack.py", line 31, in <module>
    import skimage.io
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/__init__.py", line 135, in <module>
    from .data import data_dir
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/data/__init__.py", line 13, in <module>
    from ..io import imread, use_plugin
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/io/__init__.py", line 7, in <module>
    from .manage_plugins import *
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/io/manage_plugins.py", line 24, in <module>
    from .collection import imread_collection_wrapper
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/io/collection.py", line 12, in <module>
    from ..external.tifffile import TiffFile
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/external/tifffile/__init__.py", line 1, in <module>
    from .tifffile import imsave, imread, imshow, TiffFile, TiffWriter, \
  File "/home/iimog/mambaforge/envs/starfish-issue-test/lib/python3.8/site-packages/skimage/external/tifffile/tifffile.py", line 298, in <module>
    from . import _tifffile
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe . Check the section C-API incompatibility at the Troubleshooting ImportError section at https://numpy.org/devdocs/user/troubleshooting-importerror.html#c-api-incompatibility for indications on how to solve this problem .

So far I have failed to collect a simple series of mamba and pip commands, that leads to a working starfish environment. I'd be happy to help, adding compatibility for more recent python and library versions. This is particularly important if napari is required in the same environment, because it is only compatible with relatively recent versions.

spacetx / starfish