Export AnnData error

[megleila]

Hi,

I've been processing my IMC data through steinbock without issue, but have come across one file/ROI which I continue to get the below error for when trying to create AnnData object:

C:\Users\mb2925\IMC\AIN\DriveDuplicate\Batch7>steinbock export anndata --no-concat --intensities intensities --data regionprops --neighbors neighbors -o anndata 2024-01-10 17:55:52,766 INFO steinbock - anndata/Batch7a_001.h5ad 2024-01-10 17:55:53,138 INFO steinbock - anndata/Batch7a_002.h5ad 2024-01-10 17:55:53,479 INFO steinbock - anndata/Batch7a_003.h5ad 2024-01-10 17:55:53,843 INFO steinbock - anndata/Batch7a_004.h5ad 2024-01-10 17:55:54,233 INFO steinbock - anndata/Batch7a_005.h5ad 2024-01-10 17:55:54,527 ERROR steinbock.export.data - Error creating AnnData object for image Batch7a_006.tiff; skipping image Traceback (most recent call last): File "/app/steinbock/steinbock/export/data.py", line 123, in try_convert_to_anndata_from_disk yield ( GeneratorExit Exception ignored in: <generator object try_convert_to_anndata_from_disk at 0x7fcd75ae4ba0> Traceback (most recent call last): File "/app/steinbock/steinbock/export/_cli/data.py", line 273, in anndata_cmd anndata_file = write_anndata(adata, anndata_file) RuntimeError: generator ignored GeneratorExit Traceback (most recent call last): File "/opt/venv/lib/python3.8/site-packages/anndata/_io/utils.py", line 214, in func_wrapper return func(elem, key, val, *args, *kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/specs/registry.py", line 171, in write_elem _REGISTRY.get_writer(dest_type, (t, elem.dtype.kind), modifiers)( File "/opt/venv/lib/python3.8/site-packages/anndata/_io/specs/registry.py", line 24, in wrapper result = func(g, k, args, kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/specs/methods.py", line 346, in write_vlen_string_array f.create_dataset(k, data=elem.astype(str_dtype), dtype=str_dtype, dataset_kwargs) File "/opt/venv/lib/python3.8/site-packages/h5py/_hl/group.py", line 149, in create_dataset dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds) File "/opt/venv/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 145, in make_new_dset dset_id.write(h5s.ALL, h5s.ALL, data) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5d.pyx", line 232, in h5py.h5d.DatasetID.write File "h5py/_proxy.pyx", line 145, in h5py._proxy.dset_rw File "h5py/_conv.pyx", line 444, in h5py._conv.str2vlen File "h5py/_conv.pyx", line 95, in h5py._conv.generic_converter File "h5py/_conv.pyx", line 249, in h5py._conv.conv_str2vlen TypeError: Can't implicitly convert non-string objects to strings

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/app/steinbock/steinbock/main.py", line 17, in steinbock_cmd_group(prog_name="steinbock") File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/app/steinbock/steinbock/_cli/utils.py", line 35, in wrapper return func(*args, *kwargs) File "/app/steinbock/steinbock/export/_cli/data.py", line 273, in anndata_cmd anndata_file = write_anndata(adata, anndata_file) File "/app/steinbock/steinbock/export/_cli/data.py", line 203, in write_anndata adata.write_h5ad(anndata_file) File "/opt/venv/lib/python3.8/site-packages/anndata/_core/anndata.py", line 1918, in write_h5ad _write_h5ad( File "/opt/venv/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 98, in write_h5ad write_elem(f, "obs", adata.obs, dataset_kwargs=dataset_kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/utils.py", line 214, in func_wrapper return func(elem, key, val, args, kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/specs/registry.py", line 175, in write_elem _REGISTRY.get_writer(dest_type, t, modifiers)(f, k, elem, *args, *kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/specs/registry.py", line 24, in wrapper result = func(g, k, args, **kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/specs/methods.py", line 514, in write_dataframe write_elem(group, colname, series._values, dataset_kwargs=dataset_kwargs) File "/opt/venv/lib/python3.8/site-packages/anndata/_io/utils.py", line 220, in func_wrapper raise type(e)( TypeError: Can't implicitly convert non-string objects to strings

Above error raised while writing key 'image_recovery_file' of <class 'h5py._hl.group.Group'> to /

All steps before this ran smoothly, the folder structures/file names match, and there was no issue with this run or that particular ROI. I have attached this one file in case needed. Not sure what to do -- I did try starting over and got the same error messages.

I attached a zip file with the img file and anndata file for this region, had to exclude a lot of the other data for this region due to the size limits for upload. Let me know what you think, and if you need any other file outputs to see what might be going on.

Batch7 (4).zip

[Milad4849]

Hi @megleila

Thanks for reaching out. I am afraid I cannot reproduce the error with the provided files and the output seems fine. How do you use steinbock (python package vs container, version, flavor...)?

[megleila]

I use Docker, a container. I’ve had no issues with all my other files — everything else works. Strange that you got no error; I’ve re-done this multiple times. Not sure what to do.

On Jan 17, 2024, at 11:49 AM, Milad4849 @.***> wrote:

Hi @megleila https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmegleila&data=05%7C02%7Cmegan.baker%40yale.edu%7Ca005f0ff481c445d309208dc177c508a%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C638411069896968754%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FiqNK6GqkABYNaRVd2roIxfa8szjzprNcizYMz2i8oI%3D&reserved=0 Thanks for reaching out. I am afraid I cannot reproduce the error with the provided files and the output seems fine. How do you use steinbock (python package vs container, version, flavor...)?

— Reply to this email directly, view it on GitHub https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FBodenmillerGroup%2Fsteinbock%2Fissues%2F235%23issuecomment-1896205427&data=05%7C02%7Cmegan.baker%40yale.edu%7Ca005f0ff481c445d309208dc177c508a%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C638411069896978384%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UkrFBDKpbEn4Kd7pQQYX0WdafCovghMCwiE7fJTWsRg%3D&reserved=0, or unsubscribe https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FBAH2R6LZENGAHPGOECMDKQ3YO76KVAVCNFSM6AAAAABBVE7ZIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJWGIYDKNBSG4&data=05%7C02%7Cmegan.baker%40yale.edu%7Ca005f0ff481c445d309208dc177c508a%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C638411069896986558%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mz1NsZ3TUfuxZNgtsgC6Vh%2FzCsH7BDLh8MjMSZhxJZs%3D&reserved=0. You are receiving this because you were mentioned.

[Milad4849]

Closing due to inactivity