search

openpipelines-bio / openpipeline

https://openpipelines.bio
MIT License
29 stars 14 forks source link

Error during concat process of the full pipeline v0.12.0 #623

Closed KaiWaldrant closed 1 year ago

KaiWaldrant commented 1 year ago

When running the full pipeline with public data, this error popped up (link):

Command output:
  2023-11-14 16:15:43,311 INFO     
  Concatenating data from paths:
    TianKampmann2021_CRISPRi.filter_with_scrublet.output.h5mu
    TianKampmann2019_day7neuron.filter_with_scrublet.output.h5mu
    TianKampmann2021_CRISPRa.filter_with_scrublet.output.h5mu
    TianKampmann2019_iPSC.filter_with_scrublet.output.h5mu
  2023-11-14 16:15:43,311 INFO     Concatenating samples.
  2023-11-14 16:16:45,864 INFO     Writing out modality 'rna' to 'combined_rna.concat.output_uncompressed.h5mu' with compression 'None'.

Command error:
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/registry.py", line 311, in write_elem
      return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/registry.py", line 52, in wrapper
      result = func(g, k, *args, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/methods.py", line 359, in write_vlen_string_array
      f.create_dataset(k, data=elem.astype(str_dtype), dtype=str_dtype, **dataset_kwargs)
    File "/usr/local/lib/python3.10/site-packages/h5py/_hl/group.py", line 183, in create_dataset
      dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
    File "/usr/local/lib/python3.10/site-packages/h5py/_hl/dataset.py", line 166, in make_new_dset
      dset_id.write(h5s.ALL, h5s.ALL, data)
    File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
    File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
    File "h5py/h5d.pyx", line 282, in h5py.h5d.DatasetID.write
    File "h5py/_proxy.pyx", line 147, in h5py._proxy.dset_rw
    File "h5py/_conv.pyx", line 442, in h5py._conv.str2vlen
    File "h5py/_conv.pyx", line 96, in h5py._conv.generic_converter
    File "h5py/_conv.pyx", line 247, in h5py._conv.conv_str2vlen
  TypeError: Can't implicitly convert non-string objects to strings

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File ".viash_script.sh", line 333, in <module>
      main()
    File ".viash_script.sh", line 323, in main
      concatenate_modalities(n_processes,
    File ".viash_script.sh", line 287, in concatenate_modalities
      mu.write_h5ad(output_file_uncompressed, data=new_mod, mod=mod_name)
    File "/usr/local/lib/python3.10/site-packages/mudata/_core/io.py", line 264, in write_h5ad
      write_elem(fmd, "obs", adata.obs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/registry.py", line 353, in write_elem
      Writer(_REGISTRY).write_elem(store, k, elem, dataset_kwargs=dataset_kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/utils.py", line 248, in func_wrapper
      re_raise_error(e, elem, key)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/utils.py", line 246, in func_wrapper
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/registry.py", line 311, in write_elem
      return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/registry.py", line 52, in wrapper
      result = func(g, k, *args, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/specs/methods.py", line 579, in write_dataframe
      _writer.write_elem(
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/utils.py", line 248, in func_wrapper
      re_raise_error(e, elem, key)
    File "/usr/local/lib/python3.10/site-packages/anndata/_io/utils.py", line 229, in re_raise_error
      raise type(e)(
  TypeError: Can't implicitly convert non-string objects to strings

  Above error raised while writing key 'batch' of <class 'h5py._hl.group.Group'> to /

Work dir:
  /scratch/2VwU7yVxw661Ce/52/3abc918d23bb6375e418a17e888036
KaiWaldrant commented 1 year ago

@DriesSchaumont This has already been discussed. Adding it as a ticket to track.