seung-lab / igneous

Scalable Neuroglancer compatible Downsampling, Meshing, Skeletonizing, Contrast Normalization, Transfers and more.
GNU General Public License v3.0
44 stars 17 forks source link

Mesh info file caching causing errors in MultiResShardedMeshMergeTask? #143

Closed macrintr closed 1 year ago

macrintr commented 1 year ago

I was generating sharded meshes for a small segmentation on Google Cloud Storage, and ran into this error during MultiResShardedMeshMergeTask.

File "/opt/conda/lib/python3.10/site-packages/igneous/tasks/mesh/multires.py", line 225, in MultiResShardedMeshMergeTask
    fname, shard = create_mesh_shard(
  File "/opt/conda/lib/python3.10/site-packages/igneous/tasks/mesh/multires.py", line 334, in create_mesh_shard
    meshes = {
  File "/opt/conda/lib/python3.10/site-packages/igneous/tasks/mesh/multires.py", line 335, in <dictcomp>
    label: process_mesh(
  File "/opt/conda/lib/python3.10/site-packages/igneous/tasks/mesh/multires.py", line 129, in process_mesh
    vqb = int(cv.mesh.meta.info["vertex_quantization_bits"])
KeyError: 'vertex_quantization_bits'

When I waited an hour and restarted the merge task (igneous mesh merge-sharded ...), it completed without a problem. Seems like the mesh info file is being cached at some point, but I can't figure out where (here?).

I used a fresh install of igneous from PyPI.

Test code

Create small CloudVolume

from cloudvolume import CloudVolume
info = CloudVolume.create_new_info(
    num_channels = 1, 
    layer_type="segmentation", 
    data_type="uint8", 
    encoding="raw", 
    resolution=[1,1,1], 
    voxel_offset=[0,0,1], 
    chunk_size=[32,32,32], 
    volume_size=[32,32,32])
vol = CloudVolume(PATH, info=info)
vol.commit_info()
vol[:] = 1

Generate meshes with CLI

igneous mesh forge $PATH --mip 0 --queue $QUEUE --sharded 
igneous execute $QUEUE
igneous mesh merge-sharded $PATH --queue $QUEUE --nlod 0
igneous execute $QUEUE
william-silversmith commented 1 year ago

Hi Tommy! Sorry I forgot to answer this question. I just encountered this myself and put in a fix. I'll try to release a new version of Igneous soon.

william-silversmith commented 1 year ago

This is released in 4.13.0. Thanks for reporting!