search

NeurodataWithoutBorders / pynwb

A Python API for working with Neurodata stored in the NWB Format
https://pynwb.readthedocs.io
Other
176 stars 84 forks source link

[Bug]: Can't Export NWB #1973

Open rcpeene opened 1 day ago

rcpeene commented 1 day ago

What happened?

I am appending columns to the NWB electrodes table and then exporting it as a new file, described in more detail here https://github.com/hdmf-dev/hdmf-zarr/issues/179

I receive the error below when trying to export the NWB. Is this an hdmf problem? The same code works when using hdmf-zarr to export.

Steps to Reproduce

Use pynwb and run this code where locs, xs, ys, and zs are numpy arrays

    with io_class(str(scratch_nwb_path), mode='a') as read_io:
        nwb = read_io.read()

        print('Getting new electrode columns')
        locs, xs, ys, zs = get_new_electrode_colums(nwb, ccf_map)

        nwb.electrodes.location.data[:] = np.array(locs)
        nwb.electrodes.add_column('x', 'ccf x coordinate', data=xs)
        nwb.electrodes.add_column('y', 'ccf y coordinate', data=ys)
        nwb.electrodes.add_column('z', 'ccf z coordinate', data=zs)

        print('Exporting to NWB:',result_nwb_path)
        with io_class(str(result_nwb_path), "w") as export_io:
            export_io.export(src_io=read_io, nwbfile=nwb, write_args={'link_data': False})
        print(f"Done writing {result_nwb_path}")

Traceback

Traceback (most recent call last):
  File "/code/run_capsule.py", line 126, in <module>
    if __name__ == "__main__": run()
  File "/code/run_capsule.py", line 122, in run
    export_io.export(src_io=read_io, nwbfile=nwb, write_args={'link_data': False})
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/pynwb/__init__.py", line 410, in export
    super().export(**kwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 458, in export
    super().export(**ckwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/io.py", line 163, in export
    bldr = src_io.manager.build(container, source=self.__source, root=True, export=True)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/manager.py", line 173, in build
    result = self.__type_map.build(container, self, source=source, spec_ext=spec_ext, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/manager.py", line 802, in build
    builder = obj_mapper.build(container, manager, builder=builder, source=source, spec_ext=spec_ext, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 730, in build
    self.__add_groups(builder, self.__spec.groups, container, manager, source, export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1139, in __add_groups
    self.__add_groups(sub_builder, spec.groups, container, build_manager, source, export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1152, in __add_groups
    self.__add_containers(builder, spec, attr_value, build_manager, source, container, export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1210, in __add_containers
    self.__add_containers(builder, spec, container, build_manager, source, parent_container, export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1172, in __add_containers
    new_builder = build_manager.build(value, source=source, spec_ext=spec, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/manager.py", line 173, in build
    result = self.__type_map.build(container, self, source=source, spec_ext=spec_ext, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/manager.py", line 802, in build
    builder = obj_mapper.build(container, manager, builder=builder, source=source, spec_ext=spec_ext, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 729, in build
    self.__add_datasets(builder, self.__spec.datasets, container, manager, source, export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1122, in __add_datasets
    self.__add_containers(builder, spec, attr_value, build_manager, source, container, export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 1172, in __add_containers
    new_builder = build_manager.build(value, source=source, spec_ext=spec, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/manager.py", line 173, in build
    result = self.__type_map.build(container, self, source=source, spec_ext=spec_ext, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/manager.py", line 802, in build
    builder = obj_mapper.build(container, manager, builder=builder, source=source, spec_ext=spec_ext, export=export)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/build/objectmapper.py", line 798, in build
    builder = DatasetBuilder(
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 667, in func_call
    pargs = _check_args(args, kwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 660, in _check_args
    raise ExceptionType(msg)
TypeError: DatasetBuilder.__init__: incorrect type for 'data' (got 'AstypeWrapper', expected 'ndarray, list, tuple, Dataset, Array, StrDataset, HDMFDataset, AbstractDataChunkIterator, str, int, float, bytes, bool, DataIO, DatasetBuilder, RegionBuilder, Iterable, datetime or date')

Operating System

Windows

Python Executable

Conda

Python Version

3.9

Package Versions

pynwb==2.8.2

Code of Conduct

stephprince commented 8 hours ago

Hi @rcpeene, I believe this would be an hdmf issue, but I wasn't able to reproduce the error with a small test example so I’m not positive what’s causing the error.

@mavaylon1 would you be able to take a look since you have been working on export?