NeurodataWithoutBorders / lindi

Linked Data Interface (LINDI) - cloud-friendly access to NWB data
BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

Data with shape (0,0,0) triggering assertion error after get chunk refactor #76

Closed rly closed 4 months ago

rly commented 4 months ago

I think #68 introduced a bug where now, a dataset with shape (0,0,0) as in dandiset 000409 at path acquisition/OriginalVideoBodyCamera/data raises an assertion error when generating the RFS that the chunk reference is invalid: ('https://api.dandiarchive.org/api/assets/4bab2e40-6253-4204-89ba-89b4167d46b4/download/', None, 0)

The second element should not be None. This case was previously addressed. I'll look into this later this week or if I have time today.

Error location:

File ~/Documents/NWB/lindi/lindi/LindiH5ZarrStore/LindiH5ZarrStore.py:645, in LindiH5ZarrStore.to_reference_file_system.<locals>._process_dataset(key, item)
    640 external_array_link = zattrs_dict.get(
    641     "_EXTERNAL_ARRAY_LINK", None
    642 )
    643 if external_array_link is None:
    644     # Only add chunk references for datasets without an external array link
--> 645     self._add_chunk_info_to_refs(key, _add_ref, _add_ref_chunk)

File ~/Documents/NWB/lindi/lindi/LindiH5ZarrStore/LindiH5ZarrStore.py:485, in LindiH5ZarrStore._add_chunk_info_to_refs(self, key_parent, add_ref, add_ref_chunk)
    483 byte_offset, byte_count = _get_byte_range_for_contiguous_dataset(h5_item)
    484 key_name = ".".join("0" for _ in range(h5_item.ndim))
--> 485 add_ref_chunk(f"{key_parent}/{key_name}", (self._url, byte_offset, byte_count))

File ~/Documents/NWB/lindi/lindi/LindiH5ZarrStore/LindiH5ZarrStore.py:592, in LindiH5ZarrStore.to_reference_file_system.<locals>._add_ref_chunk(key, data)
    591 def _add_ref_chunk(key: str, data: Tuple[str, int, int]):
--> 592     assert data[1] is not None

TODO: