Run names don't match between TMT-I report and annotation

akyocum commented 1 year ago

Hey there,

Now working through a multi-plex, multi-fraction, TMT experiment. The error I receive is this:

I've reviewed the documentation that shows how to manipulate the annotation file that is outputted from FragPipe and it looks like I'm ok in that regard. I checked and the column headers between the annotation.tsv and abundance_gene_MD.tsv and they do indeed match. I'm attaching the annotation file and the head of the abundance_gene_MD.tsv file for troubleshooting.

Thanks for your help!

Anastasia

experiment_annotation.txt abundance_gene_MD_4issue.txt

hsiaoyi0504 commented 1 year ago

@akyocum So the issue is that you have multiple lines of entries in the same plex,channel combination or sample_name like I could see multiple 30_UT_BP_18A in your file. Is that what directly output from FragPipe? I assume the number of entries in the automatically generated template should be the same as the number of samples in TMT-I report.

akyocum commented 1 year ago

It’s due to a fractionation experiment. Each plex was fractionated 5-times before injection into MS.

Anastasia

From: hsiao yi @.> Date: Thursday, February 9, 2023 at 12:51 PM To: Nesvilab/FragPipe-Analyst @.> Cc: Yocum, Anastasia @.>, Mention @.> Subject: Re: [Nesvilab/FragPipe-Analyst] Run names don't match between TMT-I report and annotation (Issue #8) External Email - Use Caution

@akyocumhttps://github.com/akyocum So the issue is that you have multiple lines of entries in the same plex,channel combination or sample_name like I could see multiple 30_UT_BP_18A in your file. Is that what directly output from FragPipe? I assume the number of entries in the automatically generated template should be the same as the number of samples in TMT-I report.

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe-Analyst/issues/8#issuecomment-1424587073, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A5SSSONYADMX26SZSX7T2UDWWUVDHANCNFSM6AAAAAAUVWBLYE. You are receiving this because you were mentioned.Message ID: @.***>

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

hsiaoyi0504 commented 1 year ago

@akyocum For TMT-I report, we don't separate the fractionated results. That's why I wonder if FragPipe generated that or you created it by yourself. To make it compatible, match the number of sample_name in your experimental annotation as the number of samples available in the TMT-I report.

akyocum commented 1 year ago

Sorry, I didn't answer your question directly.

Yes, this annotation file was automatically generated from FragPipe, I edited that provided annotation.txt file to have the header / columns needed per the documentation.

I too would assume that the number of entries in the template should match in the TMT-I report, but it's not the case.

In each plex directory, the psm.txt file has all files:scanNumbers representing all fractions in that specific plex. The column headers in these individual plex psm file have coalesced the fractions into one sample name. Makes sense.

Then, the combined_psm.txt file also lists as rows the scan numbers and all samples across all plexes as column headers. There are appropriately 'zeros' in the observations when that file:scanNumber (plex/fraction) is not in that sample.

Then, the TMT-I report coalesced to just have column headers to have all samples in all plexes. But the annotation file has all fractions in each plex listed out iteratively.

Maybe the path forward is to list one of the fractions and one label?

Anastasia

akyocum commented 1 year ago

Oppps.. we posted at the same time. But same conclusion.

So, for channel / plex just slot in a number?

Anastasia

hsiaoyi0504 commented 1 year ago

@akyocum could you post the manifest file you have for your FragPipe run? We kind of don't know why that would happen. For channel, that will be 126, 127N etc. For plex, you could put whatever you want, like I usually have TMT_01 for first plex-set.

akyocum commented 1 year ago

Certainly,

Removing the fraction rows so that there was one row per sample name worked however.

I feel this wasn't an optimized set up.
Experimental design is complex however, five Plex, with 5-fractions each, TMT-16 with two pooled controls. There are biological replicates, but needed to keep each individual file separate for downstream analysis.

fragpipe-files.fp-manifest.txt

hsiaoyi0504 commented 1 year ago

@akyocum May I asked which FragPipe version you are using? I don't think your experimental design is too complex. In CPTAC projects, we have similar setting that multiple fractions are collected for each samples.

akyocum commented 1 year ago

Yep,

FragPipe v 19.1

fcyu commented 1 year ago

Hi @akyocum ,

Can you also upload the log file with the name starts with log-?

Thanks,

Fengchao

akyocum commented 1 year ago

sure, thanks for your help!

log_2023-02-07_18-08-45.txt

Anastasia

fcyu commented 1 year ago

There are duplicated "samples" in the annotation.txt files, which results in "duplicated" rows in the experiment_annotation.txt file.

~~~~~~annotation files~~~~~~~
E:\Oshea_UPMC_TreatedOrganoids_phospho\mzml4FragPipe\plex1\Plex1_annotation.txt:
Channel annotation
126 60_CHIR_BP_11E
127C    60_DMSO_BP_17A
127N    60_CHIR_BP_17A
128C    60_IWR_BP_18A
128N    60_IWR_BP_19D
129C    90_UT_BP_11E
129N    90_UT_BP_17A
130C    90_DMSO_C_9G
130N    90_CHIR_C_9G
131C    90_IWR_BP_11E
131N    30_UT_BP_18A
132C    60_UT_C_10V
132N    60_UT_BP_11E
133N    60_DMSO_C_10V
133C    pool_16
134N    bridge_16
126 60_CHIR_BP_11E
127C    60_DMSO_BP_17A
127N    60_CHIR_BP_17A
128C    60_IWR_BP_18A
128N    60_IWR_BP_19D
129C    90_UT_BP_11E
129N    90_UT_BP_17A
130C    90_DMSO_C_9G
130N    90_CHIR_C_9G
131C    90_IWR_BP_11E
131N    30_UT_BP_18A
132C    60_UT_C_10V
132N    60_UT_BP_11E
133N    60_DMSO_C_10V
133C    pool_17
134N    bridge_17
126 60_CHIR_BP_11E
127C    60_DMSO_BP_17A
127N    60_CHIR_BP_17A
128C    60_IWR_BP_18A
128N    60_IWR_BP_19D
129C    90_UT_BP_11E
129N    90_UT_BP_17A
130C    90_DMSO_C_9G
130N    90_CHIR_C_9G
131C    90_IWR_BP_11E
131N    30_UT_BP_18A
132C    60_UT_C_10V
132N    60_UT_BP_11E
133N    60_DMSO_C_10V
133C    pool_18
134N    bridge_18
126 60_CHIR_BP_11E
127C    60_DMSO_BP_17A
127N    60_CHIR_BP_17A
128C    60_IWR_BP_18A
128N    60_IWR_BP_19D
129C    90_UT_BP_11E
129N    90_UT_BP_17A
130C    90_DMSO_C_9G
130N    90_CHIR_C_9G
131C    90_IWR_BP_11E
131N    30_UT_BP_18A
132C    60_UT_C_10V
132N    60_UT_BP_11E
133N    60_DMSO_C_10V
133C    pool_19
134N    bridge_19
126 60_CHIR_BP_11E
127C    60_DMSO_BP_17A
127N    60_CHIR_BP_17A
128C    60_IWR_BP_18A
128N    60_IWR_BP_19D
129C    90_UT_BP_11E
129N    90_UT_BP_17A
130C    90_DMSO_C_9G
130N    90_CHIR_C_9G
131C    90_IWR_BP_11E
131N    30_UT_BP_18A
132C    60_UT_C_10V
132N    60_UT_BP_11E
133N    60_DMSO_C_10V
133C    pool_20
134N    bridge_20
E:\Oshea_UPMC_TreatedOrganoids_phospho\mzml4FragPipe\plex2\Plex2_annotation.txt:
Channel annotation
126 90_CHIR_BP_11E
127C    90_IWR_BP_17A
127N    90_IWR_BP_18A
128C    90_UT_BP_18A
128N    90_IWR_C_8WW
129C    90_DMSO_BP_11E
129N    60_UT_BP_17A
130C    60_UT_BP_18A
130N    60_DMSO_C_5G
131C    60_IWR_C_5G
131N    60_IWR_C_8WW
132C    60_CHIR_BP_18A
132N    60_DMSO_BP_19D
133N    60_CHIR_BP_19D
133C    pool_6
134N    bridge_6
126 90_CHIR_BP_11E
127C    90_IWR_BP_17A
127N    90_IWR_BP_18A
128C    90_UT_BP_18A
128N    90_IWR_C_8WW
129C    90_DMSO_BP_11E
129N    60_UT_BP_17A
130C    60_UT_BP_18A
130N    60_DMSO_C_5G
131C    60_IWR_C_5G
131N    60_IWR_C_8WW
132C    60_CHIR_BP_18A
132N    60_DMSO_BP_19D
133N    60_CHIR_BP_19D
133C    pool_7
134N    bridge_7
126 90_CHIR_BP_11E
127C    90_IWR_BP_17A
127N    90_IWR_BP_18A
128C    90_UT_BP_18A
128N    90_IWR_C_8WW
129C    90_DMSO_BP_11E
129N    60_UT_BP_17A
130C    60_UT_BP_18A
130N    60_DMSO_C_5G
131C    60_IWR_C_5G
131N    60_IWR_C_8WW
132C    60_CHIR_BP_18A
132N    60_DMSO_BP_19D
133N    60_CHIR_BP_19D
133C    pool_8
134N    bridge_8
126 90_CHIR_BP_11E
127C    90_IWR_BP_17A
127N    90_IWR_BP_18A
128C    90_UT_BP_18A
128N    90_IWR_C_8WW
129C    90_DMSO_BP_11E
129N    60_UT_BP_17A
130C    60_UT_BP_18A
130N    60_DMSO_C_5G
131C    60_IWR_C_5G
131N    60_IWR_C_8WW
132C    60_CHIR_BP_18A
132N    60_DMSO_BP_19D
133N    60_CHIR_BP_19D
133C    pool_9
134N    bridge_9
126 90_CHIR_BP_11E
127C    90_IWR_BP_17A
127N    90_IWR_BP_18A
128C    90_UT_BP_18A
128N    90_IWR_C_8WW
129C    90_DMSO_BP_11E
129N    60_UT_BP_17A
130C    60_UT_BP_18A
130N    60_DMSO_C_5G
131C    60_IWR_C_5G
131N    60_IWR_C_8WW
132C    60_CHIR_BP_18A
132N    60_DMSO_BP_19D
133N    60_CHIR_BP_19D
133C    pool_10
134N    bridge_10
E:\Oshea_UPMC_TreatedOrganoids_phospho\mzml4FragPipe\plex3\Plex3_annotation.txt:
Channel annotation
126 30_UT_C_5G
127C    60_UT_BP_19D
127N    60_CHIR_C_5G
128C    60_DMSO_C_8WW
128N    60_CHIR_C_8WW
129C    60_IWR_C_9G
129N    60_DMSO_BP_11E
130C    90_UT_BP_19D
130N    90_DMSO_C_8WW
131C    90_IWR_C_10V
131N    90_CHIR_BP_17A
132C    90_DMSO_BP_18A
132N    90_CHIR_BP_18A
133N    90_IWR_BP_19D
133C    pool_1
134N    bridge_1
126 30_UT_C_5G
127C    60_UT_BP_19D
127N    60_CHIR_C_5G
128C    60_DMSO_C_8WW
128N    60_CHIR_C_8WW
129C    60_IWR_C_9G
129N    60_DMSO_BP_11E
130C    90_UT_BP_19D
130N    90_DMSO_C_8WW
131C    90_IWR_C_10V
131N    90_CHIR_BP_17A
132C    90_DMSO_BP_18A
132N    90_CHIR_BP_18A
133N    90_IWR_BP_19D
133C    pool_2
134N    bridge_2
126 30_UT_C_5G
127C    60_UT_BP_19D
127N    60_CHIR_C_5G
128C    60_DMSO_C_8WW
128N    60_CHIR_C_8WW
129C    60_IWR_C_9G
129N    60_DMSO_BP_11E
130C    90_UT_BP_19D
130N    90_DMSO_C_8WW
131C    90_IWR_C_10V
131N    90_CHIR_BP_17A
132C    90_DMSO_BP_18A
132N    90_CHIR_BP_18A
133N    90_IWR_BP_19D
133C    pool_3
134N    bridge_3
126 30_UT_C_5G
127C    60_UT_BP_19D
127N    60_CHIR_C_5G
128C    60_DMSO_C_8WW
128N    60_CHIR_C_8WW
129C    60_IWR_C_9G
129N    60_DMSO_BP_11E
130C    90_UT_BP_19D
130N    90_DMSO_C_8WW
131C    90_IWR_C_10V
131N    90_CHIR_BP_17A
132C    90_DMSO_BP_18A
132N    90_CHIR_BP_18A
133N    90_IWR_BP_19D
133C    pool_4
134N    bridge_4
126 30_UT_C_5G
127C    60_UT_BP_19D
127N    60_CHIR_C_5G
128C    60_DMSO_C_8WW
128N    60_CHIR_C_8WW
129C    60_IWR_C_9G
129N    60_DMSO_BP_11E
130C    90_UT_BP_19D
130N    90_DMSO_C_8WW
131C    90_IWR_C_10V
131N    90_CHIR_BP_17A
132C    90_DMSO_BP_18A
132N    90_CHIR_BP_18A
133N    90_IWR_BP_19D
133C    pool_5
134N    bridge_5
E:\Oshea_UPMC_TreatedOrganoids_phospho\mzml4FragPipe\plex4\Plex4_annotation.txt:
Channel annotation
126 90_DMSO_C_5G
127C    90_IWR_C_5G
127N    90_CHIR_BP_19D
128C    60_UT_C_5G
128N    60_CHIR_C_9G
129C    60_IWR_C_10V
129N    30_UT_C_9G
130C    30_UT_C_10V
130N    60_DMSO_BP_18A
131C    90_UT_C_5G
131N    90_UT_C_8WW
132C    90_DMSO_C_10V
132N    90_CHIR_C_10V
133N    90_DMSO_BP_17A
133C    pool_21
134N    bridge_21
126 90_DMSO_C_5G
127C    90_IWR_C_5G
127N    90_CHIR_BP_19D
128C    60_UT_C_5G
128N    60_CHIR_C_9G
129C    60_IWR_C_10V
129N    30_UT_C_9G
130C    30_UT_C_10V
130N    60_DMSO_BP_18A
131C    90_UT_C_5G
131N    90_UT_C_8WW
132C    90_DMSO_C_10V
132N    90_CHIR_C_10V
133N    90_DMSO_BP_17A
133C    pool_22
134N    bridge_22
126 90_DMSO_C_5G
127C    90_IWR_C_5G
127N    90_CHIR_BP_19D
128C    60_UT_C_5G
128N    60_CHIR_C_9G
129C    60_IWR_C_10V
129N    30_UT_C_9G
130C    30_UT_C_10V
130N    60_DMSO_BP_18A
131C    90_UT_C_5G
131N    90_UT_C_8WW
132C    90_DMSO_C_10V
132N    90_CHIR_C_10V
133N    90_DMSO_BP_17A
133C    pool_23
134N    bridge_23
126 90_DMSO_C_5G
127C    90_IWR_C_5G
127N    90_CHIR_BP_19D
128C    60_UT_C_5G
128N    60_CHIR_C_9G
129C    60_IWR_C_10V
129N    30_UT_C_9G
130C    30_UT_C_10V
130N    60_DMSO_BP_18A
131C    90_UT_C_5G
131N    90_UT_C_8WW
132C    90_DMSO_C_10V
132N    90_CHIR_C_10V
133N    90_DMSO_BP_17A
133C    pool_24
134N    bridge_24
126 90_DMSO_C_5G
127C    90_IWR_C_5G
127N    90_CHIR_BP_19D
128C    60_UT_C_5G
128N    60_CHIR_C_9G
129C    60_IWR_C_10V
129N    30_UT_C_9G
130C    30_UT_C_10V
130N    60_DMSO_BP_18A
131C    90_UT_C_5G
131N    90_UT_C_8WW
132C    90_DMSO_C_10V
132N    90_CHIR_C_10V
133N    90_DMSO_BP_17A
133C    pool_25
134N    bridge_25
E:\Oshea_UPMC_TreatedOrganoids_phospho\mzml4FragPipe\plex5\Plex5_annotation.txt:
Channel annotation
126 90_CHIR_C_8WW
127C    90_IWR_C_9G
127N    90_DMSO_BP_19D
128C    30_UT_BP_11E
128N    30_UT_BP_17A
129C    60_UT_C_8WW
129N    60_UT_C_9G
130C    60_DMSO_C_9G
130N    60_CHIR_C_10V
131C    60_IWR_BP_11E
131N    60_IWR_BP_17A
132C    90_UT_C_9G
132N    90_UT_C_10V
133N    90_CHIR_C_5G
133C    pool_11
134N    bridge_11
126 90_CHIR_C_8WW
127C    90_IWR_C_9G
127N    90_DMSO_BP_19D
128C    30_UT_BP_11E
128N    30_UT_BP_17A
129C    60_UT_C_8WW
129N    60_UT_C_9G
130C    60_DMSO_C_9G
130N    60_CHIR_C_10V
131C    60_IWR_BP_11E
131N    60_IWR_BP_17A
132C    90_UT_C_9G
132N    90_UT_C_10V
133N    90_CHIR_C_5G
133C    pool_12
134N    bridge_12
126 90_CHIR_C_8WW
127C    90_IWR_C_9G
127N    90_DMSO_BP_19D
128C    30_UT_BP_11E
128N    30_UT_BP_17A
129C    60_UT_C_8WW
129N    60_UT_C_9G
130C    60_DMSO_C_9G
130N    60_CHIR_C_10V
131C    60_IWR_BP_11E
131N    60_IWR_BP_17A
132C    90_UT_C_9G
132N    90_UT_C_10V
133N    90_CHIR_C_5G
133C    pool_13
134N    bridge_13
126 90_CHIR_C_8WW
127C    90_IWR_C_9G
127N    90_DMSO_BP_19D
128C    30_UT_BP_11E
128N    30_UT_BP_17A
129C    60_UT_C_8WW
129N    60_UT_C_9G
130C    60_DMSO_C_9G
130N    60_CHIR_C_10V
131C    60_IWR_BP_11E
131N    60_IWR_BP_17A
132C    90_UT_C_9G
132N    90_UT_C_10V
133N    90_CHIR_C_5G
133C    pool_14
134N    bridge_14
126 90_CHIR_C_8WW
127C    90_IWR_C_9G
127N    90_DMSO_BP_19D
128C    30_UT_BP_11E
128N    30_UT_BP_17A
129C    60_UT_C_8WW
129N    60_UT_C_9G
130C    60_DMSO_C_9G
130N    60_CHIR_C_10V
131C    60_IWR_BP_11E
131N    60_IWR_BP_17A
132C    90_UT_C_9G
132N    90_UT_C_10V
133N    90_CHIR_C_5G
133C    pool_15
134N    bridege_15
~~~~~~~~~~~~~~~~~~~~~~

Best,

Fengchao

akyocum commented 1 year ago

Ok, I think this makes more sense to me. I was creating annotations like MSstats requires where each fraction in each plex would be listed. I've removed all the duplicates and rerunning now.

hsiaoyi0504 commented 1 year ago

@akyocum I will close this now. Please let us know if you still meet some issues. Thanks.

Nesvilab / FragPipe-Analyst

Run names don't match between TMT-I report and annotation #8