`torchdata` import error with M3 Mac

[dtebaykin]

Please refer to our FAQ and look at our known issues before opening a bug report.

Describe the bug

The latest release of ark-analysis==0.7.1 errors out on the first import of notebook 1:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[1], line 7
      5 from alpineer import io_utils
      6 from skimage import io
----> 7 from ark.segmentation import marker_quantification, segmentation_utils
      8 from ark.utils import deepcell_service_utils, example_dataset, plot_utils

File ~/Documents/Bendall lab projects/ark-analysis/src/ark/segmentation/marker_quantification.py:13
     10 from skimage.measure import regionprops, regionprops_table
     12 import ark.settings as settings
---> 13 from ark.segmentation import segmentation_utils
     14 from ark.segmentation.regionprops_extraction import REGIONPROPS_FUNCTION
     15 from ark.segmentation.signal_extraction import EXTRACTION_FUNCTION

File ~/Documents/Bendall lab projects/ark-analysis/src/ark/segmentation/segmentation_utils.py:12
      9 from skimage.segmentation import find_boundaries
     11 import ark.settings as settings
---> 12 from ark.utils import plot_utils
     15 def find_nuclear_label_id(nuc_segmentation_labels, cell_coords):
     16     """Get the ID of the nuclear mask which has the greatest amount of overlap with a given cell
     17 
     18     Args:
   (...)
     27             If no matches found, returns None.
     28     """

File ~/Documents/Bendall lab projects/ark-analysis/src/ark/utils/plot_utils.py:31
     29 from ark import settings
     30 from skimage.segmentation import find_boundaries
---> 31 from ark.utils.data_utils import (
     32     ClusterMaskData,
     33     erode_mask,
     34     generate_cluster_mask,
     35     save_fov_mask,
     36     map_segmentation_labels,
     37 )
     40 @dataclass
     41 class MetaclusterColormap:
     42     """
     43     A dataclass which contains the colormap-related information for the metaclusters.
     44 
     45     """

File ~/Documents/Bendall lab projects/ark-analysis/src/ark/utils/data_utils.py:25
     23 from anndata.experimental import AnnCollection
     24 from anndata.experimental.multi_files._anncollection import ConvertType
---> 25 from torchdata.datapipes.iter import IterDataPipe
     26 from typing import Iterator, Optional
     27 try:

File /opt/miniconda3/envs/ark_env/lib/python3.10/site-packages/torchdata/datapipes/__init__.py:11
      7 from torch.utils.data import DataChunk, functional_datapipe
      9 from torchdata import _extension  # noqa: F401
---> 11 from . import iter, map, utils
     13 __all__ = ["DataChunk", "functional_datapipe", "iter", "map", "utils"]
     16 from torchdata import deprecation_warning

File /opt/miniconda3/envs/ark_env/lib/python3.10/site-packages/torchdata/datapipes/iter/__init__.py:79
     68 from torchdata.datapipes.iter.transform.callable import (
     69     BatchAsyncMapperIterDataPipe as BatchAsyncMapper,
     70     BatchMapperIterDataPipe as BatchMapper,
   (...)
     76     ThreadPoolMapperIterDataPipe as ThreadPoolMapper,
     77 )
     78 from torchdata.datapipes.iter.util.bz2fileloader import Bz2FileLoaderIterDataPipe as Bz2FileLoader
---> 79 from torchdata.datapipes.iter.util.cacheholder import (
     80     EndOnDiskCacheHolderIterDataPipe as EndOnDiskCacheHolder,
     81     InMemoryCacheHolderIterDataPipe as InMemoryCacheHolder,
     82     OnDiskCacheHolderIterDataPipe as OnDiskCacheHolder,
     83 )
     84 from torchdata.datapipes.iter.util.combining import (
     85     IterKeyZipperIterDataPipe as IterKeyZipper,
     86     MapKeyZipperIterDataPipe as MapKeyZipper,
     87     RoundRobinDemultiplexerIterDataPipe as RoundRobinDemultiplexer,
     88     UnZipperIterDataPipe as UnZipper,
     89 )
     90 from torchdata.datapipes.iter.util.cycler import CyclerIterDataPipe as Cycler, RepeaterIterDataPipe as Repeater

File /opt/miniconda3/envs/ark_env/lib/python3.10/site-packages/torchdata/datapipes/iter/util/cacheholder.py:24
     21 except ImportError:
     22     portalocker = None
---> 24 from torch.utils._import_utils import dill_available
     25 from torch.utils.data.datapipes.utils.common import _check_unpickable_fn
     27 from torch.utils.data.graph import traverse_dps

ModuleNotFoundError: No module named 'torch.utils._import_utils'

Expected behavior

torchdata should import correctly.

To Reproduce

Run notebook 1 on an M3 Mac with the current ark-analysis installation.

[srivarra]

Duplicate of #1150, will get this fixed soon.

[dtebaykin]

Tried to install ark-analysis on a Windows laptop yesterday with a collaborator, ran into the same problem. Installation steps followed: 1) Git clone ark-analysis 2) cd ark-analysis 3) Create conda environment using: conda env create -f environment.yml 4) Conda activate ark_env 5) .\start_jupyter.sh

The import code block of the first notebook errors out with the above torch.utils module error. Installing the 0.7.0 ark version solves the immediate problem (!pip install --no-cache-dir "ark-analysis==0.7.0").