search

MouseLand / suite2p

cell detection in calcium imaging recordings
http://www.suite2p.org
GNU General Public License v3.0
348 stars 240 forks source link

BUG: Having trouble reading the nwb files #1048

Closed R7anZ7 closed 10 months ago

R7anZ7 commented 11 months ago

Describe the issue:

When I tried to run the pipeline on a nwb file, it gave me the error.

The nwb data I am using right now is from :

https://dandiarchive.org/dandiset/000350/0.221219.1506/files?location=sub-20170111-1

The structure looks like :

TwoPhotonSeries data dimension imaging_plane NeuronOpticalChannel description emission_lambda description device excitation_lambda indicator location reference_frame timestamps

It contains data with the Shape: (10050, 888, 2048, 30) 10500 means the timestamps and the 30 represents the plane dimension.

Reproduce the code example:

from pathlib import Path
import numpy as np
import suite2p
import matplotlib.pyplot as plt

ops = suite2p.default_ops()
ops['delete_bin'] = True

db = {
    'data_path': ['/data1/ryan/project/benchmarking/voluseg_data1'],
    'nwb_file': "/data1/ryan/project/benchmarking/voluseg_data1/sub-20170111-1_ses-20170111T123355_ophys.nwb",
    # 'nwb_series':"TwoPhotonSeries",
    'save_path0':'/data1/ryan/project/benchmarking/data1_test/output',

}

output_ops = suite2p.run_s2p(ops=ops, db=db)

def save_plot(output_path):
    fig.tight_layout()  
    plt.savefig(output_path, dpi=300)
    plt.close()

output_dir = Path('./output_images')
output_dir.mkdir(parents=True, exist_ok=True)

fig, axes = plt.subplots(1, 4, figsize=(16, 4))

axes[0].imshow(output_ops['refImg'], cmap='gray')
axes[0].set_title("Reference Image for Registration")

axes[1].imshow(output_ops['max_proj'], cmap='gray')
axes[1].set_title("Registered Image, Max Projection")

axes[2].imshow(output_ops['meanImg'], cmap='gray')
axes[2].set_title("Mean registered image")

axes[3].imshow(output_ops['meanImgE'], cmap='gray')
axes[3].set_title("High-pass filtered Mean registered image")

save_plot(output_dir / 'figure1.png')

stats_file = Path(output_ops['save_path']).joinpath('stat.npy')
iscell = np.load(Path(output_ops['save_path']).joinpath('iscell.npy'), allow_pickle=True)[:, 0].astype(bool)
stats = np.load(stats_file, allow_pickle=True)

im = suite2p.ROI.stats_dicts_to_3d_array(stats, Ly=output_ops['Ly'], Lx=output_ops['Lx'], label_id=True)
im[im == 0] = np.nan

fig, axes = plt.subplots(1, 4, figsize=(16, 4))

axes[0].imshow(output_ops['max_proj'], cmap='gray')
axes[0].set_title("Registered Image, Max Projection")

axes[1].imshow(np.nanmax(im, axis=0), cmap='jet')
axes[1].set_title("All ROIs Found")

axes[2].imshow(np.nanmax(im[~iscell], axis=0), cmap='jet')
axes[2].set_title("All Non-Cell ROIs")

axes[3].imshow(np.nanmax(im[iscell], axis=0), cmap='jet')
axes[3].set_title("All Cell ROIs")

save_plot(output_dir / 'figure2.png')

Error message:

Traceback (most recent call last):
  File "/fs1//data1_test/setup_tiff.py", line 30, in <module>
    output_ops = suite2p.run_s2p(ops=ops, db=db)
  File "/home/zhuy37/.conda/envs/suite2p/lib/python3.9/site-packages/suite2p/run_s2p.py", line 472, in run_s2p
    ops0 = convert_funs[ops["input_format"]](ops.copy())
  File "/home/zhuy37/.conda/envs/suite2p/lib/python3.9/site-packages/suite2p/io/nwb.py", line 87, in nwb_to_binary
    nwbfile = fio.read()
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/pynwb/__init__.py", line 304, in read
    file = super().read(**kwargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 479, in read
    return super().read(**kwargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/backends/io.py", line 60, in read
    container = self.__manager.construct(f_builder)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/manager.py", line 284, in construct
    result = self.__type_map.construct(builder, self, None)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/manager.py", line 795, in construct
    return obj_mapper.construct(builder, build_manager, parent)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1228, in construct
    subspecs = self.__get_subspec_values(builder, self.spec, manager)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1157, in __get_subspec_values
    self.__get_sub_builders(groups, spec.groups, manager, ret)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1208, in __get_sub_builders
    ret.update(self.__get_subspec_values(sub_builder, subspec, manager))
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1157, in __get_subspec_values
    self.__get_sub_builders(groups, spec.groups, manager, ret)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1200, in __get_sub_builders
    sub_builder = self.__flatten(sub_builder, subspec, manager)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1213, in __flatten
    tmp = [manager.construct(b) for b in sub_builder]
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1213, in <listcomp>
    tmp = [manager.construct(b) for b in sub_builder]
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/manager.py", line 280, in construct
    result = self.__type_map.construct(builder, self, parent)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/manager.py", line 795, in construct
    return obj_mapper.construct(builder, build_manager, parent)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1228, in construct
    subspecs = self.__get_subspec_values(builder, self.spec, manager)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1157, in __get_subspec_values
    self.__get_sub_builders(groups, spec.groups, manager, ret)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1186, in __get_sub_builders
    for parent_dt in manager.namespace_catalog.get_hierarchy(ns, dt):
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/utils.py", line 664, in func_call
    return func(args[0], **pargs)
  File "/home/zhuy37/.local/lib/python3.9/site-packages/hdmf/spec/namespace.py", line 328, in get_hierarchy
    raise KeyError("'%s' not a namespace" % namespace)
KeyError: "'ndx-events' not a namespace"

Version information:

suite2p v0.14.2.dev8+gd530bad

Context for the issue:

No response

chriski777 commented 10 months ago

Hi @R7anZ7, this seems to be an issue on NWB's end. Seems there's a related issue on their repo here. Could you try following the instructions in that link and then try again?

chriski777 commented 10 months ago

Hi @R7anZ7, just following up on this issue. Has the suggestion above helped?

chriski777 commented 10 months ago

@R7anZ7, I'm closing this due to inactivity. Feel free to reopen if you've tried the suggestion above and the problem persists.