seung-lab / cloud-volume

Read and write Neuroglancer datasets programmatically.
https://twitter.com/thundercloudvol
BSD 3-Clause "New" or "Revised" License
131 stars 47 forks source link

`vol.mesh.exists('4650')` tells me meshes 4, 6, 5, and 0 don't exist :thinking: #620

Closed jasper-tms closed 2 months ago

jasper-tms commented 2 months ago

I just had this surprising experience:

In [1]: from cloudvolume import CloudVolume
In [2]: vol = CloudVolume('gs://lee-lab_female-adult-nerve-cord/alignmentV4/volume_meshes')

In [3]: vol.mesh.exists(1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[3], line 1
----> 1 vol.mesh.exists(1)

File ~/.virtualenvs/cloudvolume/lib/python3.10/site-packages/cloudvolume/datasource/precomputed/mesh/unsharded.py:108, in UnshardedLegacyPrecomputedMeshSource.exists(self, segids, progress)
    102 def exists(self, segids, progress=None):
    103   """
    104   Checks if the mesh exists.
    105 
    106   Returns: { label: path or None, ... }
    107   """
--> 108   manifest_paths = [ self.manifest_path(segid) for segid in segids ]
    109   progress = progress if progress is not None else self.config.progress
    111   cf = CloudFiles(
    112     self.meta.cloudpath, 
    113     progress=progress, 
    114     green=self.config.green, 
    115     secrets=self.config.secrets
    116   )

TypeError: 'int' object is not iterable

In [4]: vol.mesh.exists('1')
Exists: 100%|█████████████████████████████████████████████| 1/1 [00:00<00:00,  6.99it/s]
Out[4]: {'1': 'meshes/1:0'}

In [5]: vol.mesh.exists('4650')
Exists: 100%|█████████████████████████████████████████████| 4/4 [00:00<00:00, 26.92it/s]
Out[5]: {'4': None, '6': None, '5': None, '0': None}

In [6]: vol.mesh.exists(['4650'])
Exists: 100%|█████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  7.81it/s]
Out[6]: {'4650': 'meshes/4650:0'}

A few suggested changes would be:

  1. Accept integer arguments
  2. If a string argument is provided, don't iterate through it, but treat it as a single mesh ID
  3. I was a bit surprised that the returned dictionary keys are strings and not ints -- can meshes be named by non-integer (arbitrary string) values? If not I would find it more intuitive to have the keys of the returned dictionary be ints, but I understand that this would be a breaking change so you probably don't want to do it.

Thanks a lot!

william-silversmith commented 2 months ago

Hi Jasper,

This looks like a bug. Usually I call toiter on input so you can provide scalar or list inputs, but I forgot. I'll release a fix shortly, but in the meantime, you can do the following:

vol.mesh.exists([ 1 ])

william-silversmith commented 2 months ago

This should be fixed in the newest release. I fixed it for sharded meshes too.