'No objects to concatenate' error when generating a csv with all the acquisition metadata

[maxgrigori]

Hi all! I am getting a 'No objects to concatenate' error when trying to use the pipeline. I have already checked the zipped .mcd-files and the panel-.csv-file multiple times and also the directories but I just cannot resolve the issue... I would really appreciate the help! Thank you very much!

Below I have attached the respective screenshots. Following is the error text:

Generate a csv with all the acquisition metadata

The csv file can be found in the data/cpout folder

exportacquisitioncsv.export_acquisition_csv(folder_ome, fol_out=folder_cp)

---

ValueError Traceback (most recent call last)

in ----> 1 exportacquisitioncsv.export_acquisition_csv(folder_ome, fol_out=folder_cp) ~\anaconda3\envs\imctools\lib\site-packages\imctools\scripts\exportacquisitioncsv.py in export_acquisition_csv(fol_ome, fol_out, outname) 34 if outname is None: 35 outname=AC_META ---> 36 dat_meta = read_acmeta(fol_ome) 37 dat_meta.to_csv(os.path.join(fol_out, outname+SUF_CSV), index=False) 38 ~\anaconda3\envs\imctools\lib\site-packages\imctools\scripts\exportacquisitioncsv.py in read_acmeta(fol_ome) 28 29 def read_acmeta(fol_ome): ---> 30 dat_acmeta = _read_and_concat(fol_ome, SUFFIX_ACMETA, COL_ACID) 31 return dat_acmeta 32 ~\anaconda3\envs\imctools\lib\site-packages\imctools\scripts\exportacquisitioncsv.py in _read_and_concat(fol_ome, suffix, idname) 21 ac_names = os.listdir(fol_ome) 22 dat = pd.concat([pd.read_csv(os.path.join(fol_ome, a, a+suffix)) for a in ac_names], ---> 23 keys=ac_names, names=[COL_ACSESSION, VAL_TEMP]) 24 dat = dat.reset_index(VAL_TEMP, drop=True) 25 dat = dat.reset_index() ~\anaconda3\envs\imctools\lib\site-packages\pandas\core\reshape\concat.py in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy) 282 verify_integrity=verify_integrity, 283 copy=copy, --> 284 sort=sort, 285 ) 286 ~\anaconda3\envs\imctools\lib\site-packages\pandas\core\reshape\concat.py in __init__(self, objs, axis, join, keys, levels, names, ignore_index, verify_integrity, copy, sort) 329 330 if len(objs) == 0: --> 331 raise ValueError("No objects to concatenate") 332 333 if keys is None: ValueError: No objects to concatenate  

[votti]

Hey! I am currently in holidays for another week. I cannot see why this fails - did the generation of the subfolders in the folder data/ome work?

Until this is figured out you can also skip this step - it is likely not that essenitai.

[maxgrigori]

Hello Votti!

Thank You for your reply! It does generate all of the subfolders during the previous step. At least I did not find anything missing.

I have also tried to just continue with pipeline and skipping the step, however it fails to generate the ometiff-files and the folders stay empty...

Thank You for your help Best Max

[votti]

Ah so the problem is the OME files are missing?

Can you show the output printed at the previous step?

Convert mcd containing folders into imc zip folders
In [10]:

The issue seems to be with converting the zipped mcd to ome tiff in this step.

[maxgrigori]

Ok, alright. So this is what is showing:

[votti]

Alright: this means the script doesn't find the .zip folders with the mcd.

Can you check if the path is set correctly? Beginning of the script this line needs to be adapted:

folders = ['../example_data']

On Mon, Nov 2, 2020, 14:16 maxgrigori notifications@github.com wrote:

Ok, alright. So this is what is showing: [image: generate imc zip] https://user-images.githubusercontent.com/73642795/97872075-b437b300-1d15-11eb-954a-8d6dfd16816c.jpg

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BodenmillerGroup/ImcSegmentationPipeline/issues/52#issuecomment-720465547, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4R2KE32IWBLRM3BEEN57LSN2WK7ANCNFSM4TDRV5FA .

[maxgrigori]

Checked it right now again, but it does not change the result. I had also checked the directory multiple times already... Cannot find an obvious mistake.

[maxgrigori]

Alternatively, while trying around and when removing "_", I am now getting following error when trying to convert the mcd into the imc zip folders... Might this be the source of the problem?

Thank You a lot for your help! Best Max

[maxgrigori]

Hello again, I am very sorry, as I do not have any programming experience, but I just cannot work around the issue in this step... I have found the reasons for the previous errors and now the pipeline does find the .zip-files with the .mcd. However now "OSError: [Errno 22] Invalid argument: '../../data\ometiff\20201026_SK_MG_UC_biopsies_6292-1\20201026_SK_MG_UC_biopsies_6292-1_s0_slide""' is raised and I cannot pass it.

Thanks a lot for your help! Best, Max

[votti]

Hi Max, Now I am back from holidays :) So I see you managed to get a compatible ZIP folder. From the error message I read that something in the your metadata is different than the '.mcd' files we usually get which causes this issue. It is a bit an issue that the '.mcd' format is not extremely well defined, so it can look a bit different between different versions of the machine :/

We are currently updating the way we are reading the .mcd files. Could you maybe try this more 'cutting edge' version of the pipeline that uses our new file converter? https://github.com/BodenmillerGroup/ImcSegmentationPipeline/tree/imctools2 This requires that you reinstall the conda environment using the setup/conda_imctools.yml from that version of the repository.

If this again gives this error - would you be able to share the file such that we can adapt our converter (imctools) to support it as well?

Sorry that this is workiung so un-smooth for you!

[maxgrigori]

Hello Votti,

I have installed the imctools2 version and also the updated kernel and now it seems to work! Thank You so much! You have helped me a ton!

Best, Max

[AnanyaBhalla]

Hi, faced a similar issue but this link appears to be broken: https://github.com/BodenmillerGroup/ImcSegmentationPipeline/tree/imctools2

[nilseling]

Hi @AnanyaBhalla

this is a very old (3 years) issue and the pipeline has been updated in the meantime. We are also not using imctools or imctools2 (I don't even know what that is) anymore. Please use the newest version of the pipeline and raise an issue if you are having problems.

[AnanyaBhalla]

Hi @AnanyaBhalla

this is a very old (3 years) issue and the pipeline has been updated in the meantime. We are also not using imctools or imctools2 (I don't even know what that is) anymore. Please use the newest version of the pipeline and raise an issue if you are having problems.

Apologies, there seems to be this website (scam?) that suggests that these changes are recent as of today (13th November, 2023): https://githubhelp.com/bodenmillergroup/imcsegmentationpipeline/issues/52

[nilseling]

Hmm, that doesn't seem right as the issue was closed in October 2020