GNPS2 FEATURE BASED MOLECULAR NETWORKING Not Working

andrekind17 commented 10 months ago

Hi, the GNPS2 FBMN is not working. It fails almost immediately with the following error (from Nextflow workflow report). Any idea on how to solve it? Thanks

N E X T F L O W ~ version 21.10.0 Launching /app/workflows_user/feature_based_molecular_networking_workflow/nf_workflow.nf [magical_curie] - revision: afa4029ddf [- ] process > quantification_table_reform... - [- ] process > filter_spectra -

[- ] process > quantification_table_reform... - [- ] process > filter_spectra - [- ] process > librarySearchData - [- ] process > librarymergeResults -

[- ] process > quantification_table_reform... - [- ] process > filter_spectra - [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork -

[- ] process > quantification_table_reform... - [- ] process > filter_spectra - [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [- ] process > createMetadataFile - [- ] process > calculateGroupings - [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML -

[- ] process > quantification_table_reform... [ 0%] 0 of 1 [- ] process > filter_spectra - [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [- ] process > createMetadataFile [ 0%] 0 of 1 [- ] process > calculateGroupings - [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML -

executor > local (2) [cd/98b8f0] process > quantification_table_reform... [ 0%] 0 of 1 [- ] process > filter_spectra - [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [ec/c62371] process > createMetadataFile [ 0%] 0 of 1 [- ] process > calculateGroupings - [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML -

executor > local (2) [cd/98b8f0] process > quantification_table_reform... [ 0%] 0 of 1 [- ] process > filter_spectra - [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [ec/c62371] process > createMetadataFile [100%] 1 of 1 ✔ [- ] process > calculateGroupings - [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML -

executor > local (3) [cd/98b8f0] process > quantification_table_reform... [100%] 1 of 1 ✔ [57/fa96f7] process > filter_spectra (1) [ 0%] 0 of 1 [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [ec/c62371] process > createMetadataFile [100%] 1 of 1 ✔ [- ] process > calculateGroupings [ 0%] 0 of 1 [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML -

executor > local (4) [cd/98b8f0] process > quantification_table_reform... [100%] 1 of 1 ✔ [57/fa96f7] process > filter_spectra (1) [ 0%] 0 of 1 [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [ec/c62371] process > createMetadataFile [100%] 1 of 1 ✔ [f2/1a8593] process > calculateGroupings (1) [ 0%] 0 of 1 [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML -

executor > local (4) [cd/98b8f0] process > quantification_table_reform... [100%] 1 of 1 ✔ [57/fa96f7] process > filter_spectra (1) [ 0%] 0 of 1 [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [ec/c62371] process > createMetadataFile [100%] 1 of 1 ✔ [f2/1a8593] process > calculateGroupings (1) [ 0%] 0 of 1 [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML - Error executing process > 'calculateGroupings (1)'

Caused by: Process calculateGroupings (1) terminated with an error exit status (1)

Command executed:

python /app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py featuretable_reformated.csv merged_metadata.tsv clustersummary_with_groups.tsv

Command exit status: 1

Command output: filename ... ATTRIBUTE_HALO_DIMENSION_mm_EC 0 48212.mzML ... NaN 1 48213.mzML ... NaN 2 48214.mzML ... 4.0 3 48215.mzML ... 4.0 4 48231.mzML ... 12.0 ... ... ... ... 12219 NaN ... NaN 12220 NaN ... NaN 12221 NaN ... NaN 12222 NaN ... NaN 12223 NaN ... NaN

[12224 rows x 18 columns]

Command error: Traceback (most recent call last): File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 127, in main() File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 116, in main clustersummary_df = calculate_groups_metadata(feature_table_df, metadata_df) File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 49, in calculate_groups_metadata metadata_df["filename"] = metadata_df["filename"].apply(lambda x: os.path.basename(x).rstrip()) File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/site-packages/pandas/core/series.py", line 4626, in apply return SeriesApply(self, func, convert_dtype, args, kwargs).apply() File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/site-packages/pandas/core/apply.py", line 1025, in apply return self.apply_standard() File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/site-packages/pandas/core/apply.py", line 1076, in apply_standard mapped = lib.map_infer( File "pandas/_libs/lib.pyx", line 2834, in pandas._libs.lib.map_infer File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 49, in metadata_df["filename"] = metadata_df["filename"].apply(lambda x: os.path.basename(x).rstrip()) File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/posixpath.py", line 142, in basename p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not float

Work dir: /data/nf_data/server/nf_work/154a89144c4643f18329aacd3a93cffc/f2/1a85936c4405ba82ea0b3bb739bd43

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

executor > local (4) [cd/98b8f0] process > quantification_table_reform... [100%] 1 of 1 ✔ [57/fa96f7] process > filter_spectra (1) [ 0%] 0 of 1 [- ] process > librarySearchData - [- ] process > librarymergeResults - [- ] process > librarygetGNPSAnnotations - [- ] process > networkingGNPSPrepParams - [- ] process > calculatePairs - [- ] process > filterNetwork - [ec/c62371] process > createMetadataFile [100%] 1 of 1 ✔ [f2/1a8593] process > calculateGroupings (1) [100%] 1 of 1, failed: 1 ✘ [- ] process > enrichClusterSummary - [- ] process > createNetworkGraphML - Error executing process > 'calculateGroupings (1)'

Caused by: Process calculateGroupings (1) terminated with an error exit status (1)

Command executed:

python /app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py featuretable_reformated.csv merged_metadata.tsv clustersummary_with_groups.tsv

Command exit status: 1

Command output: filename ... ATTRIBUTE_HALO_DIMENSION_mm_EC 0 48212.mzML ... NaN 1 48213.mzML ... NaN 2 48214.mzML ... 4.0 3 48215.mzML ... 4.0 4 48231.mzML ... 12.0 ... ... ... ... 12219 NaN ... NaN 12220 NaN ... NaN 12221 NaN ... NaN 12222 NaN ... NaN 12223 NaN ... NaN

[12224 rows x 18 columns]

Command error: Traceback (most recent call last): File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 127, in main() File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 116, in main clustersummary_df = calculate_groups_metadata(feature_table_df, metadata_df) File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 49, in calculate_groups_metadata metadata_df["filename"] = metadata_df["filename"].apply(lambda x: os.path.basename(x).rstrip()) File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/site-packages/pandas/core/series.py", line 4626, in apply return SeriesApply(self, func, convert_dtype, args, kwargs).apply() File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/site-packages/pandas/core/apply.py", line 1025, in apply return self.apply_standard() File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/site-packages/pandas/core/apply.py", line 1076, in apply_standard mapped = lib.map_infer( File "pandas/_libs/lib.pyx", line 2834, in pandas._libs.lib.map_infer File "/app/workflows_user/feature_based_molecular_networking_workflow/bin/scripts/group_abundances.py", line 49, in metadata_df["filename"] = metadata_df["filename"].apply(lambda x: os.path.basename(x).rstrip()) File "/app/launchserver/env/conda_envs/conda_env-4e7614d066e4c2282297a9aa547f4051/lib/python3.8/posixpath.py", line 142, in basename p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not float

Work dir: /data/nf_data/server/nf_work/154a89144c4643f18329aacd3a93cffc/f2/1a85936c4405ba82ea0b3bb739bd43

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

WARN: Killing pending tasks (1)

mwang87 commented 10 months ago

Can you include the GNPS2 task?

andrekind17 commented 9 months ago

https://gnps2.org/status?task=154a89144c4643f18329aacd3a93cffc

Oezlem49 commented 4 months ago

Hi I'm trying to run but end up always with the same error. I have the same problem, what was the solution?

andrekind17 commented 4 months ago

Hi I'm trying to run but end up always with the same error. I have the same problem, what was the solution?

Hi, in the end I switched back to GNPS1. But I figured out that at that time the problem was related to the server not being able to correctly handle the metadata file (.txt) that you provide. If you run without metadata it works fine, if you add the metadata you have to make sure that the bottom part of the txt file does not contain blank spaces and/or tabs. In my case it did, so I removed all the blank spaces and it worked. It was a bit empirical I would say, I don't know if they fixed it now.

mwang87 commented 4 months ago

I think shoudl be fixed as Nov was a long time ago with a lot of fixes in between. Can't test this task:

https://gnps2.org/status?task=154a89144c4643f18329aacd3a93cffc

Since data got moved. Let us know if you have any issues anymore.

andrekind17 commented 4 months ago

I think shoudl be fixed as Nov was a long time ago with a lot of fixes in between. Can't test this task:

https://gnps2.org/status?task=154a89144c4643f18329aacd3a93cffc

Since data got moved. Let us know if you have any issues anymore.

I did it again, it failed:

https://gnps2.org/status?task=83537f3f19fe4f9f9b02979e12f4c8eb

mwang87 commented 4 months ago

Thanks!

Fixed in latest release;

https://gnps2.org/status?task=2764b6ddad5644158b79fbd330e93bb6

Wang-Bioinformatics-Lab / GNPS2_Workflows

GNPS2 FEATURE BASED MOLECULAR NETWORKING Not Working #43