Nesvilab / FragPipe

A cross-platform proteomics data analysis suite
http://fragpipe.nesvilab.org
Other
184 stars 37 forks source link

[0:00] Loading spectral library library.tsv cannot read the file ERROR: src/diann.cpp: 17316: Cannot load spectral library Process 'DIA-NN' finished, exit code: 255 #687

Closed JensSettelmeier closed 2 years ago

JensSettelmeier commented 2 years ago

log_2022-05-23_11-06-04_cannot_read_library_tsv.txt

Dear Nesvilab,

I have issues using the "DIA_SpecLib_Quant" workflow of FragPipe v17.1 (logfile attached). As far as I understood during the MSFragger step, a spectral library should be generated and later used by DIA-NN. And it seems like it is this file corresponding to the spectral library which can not be read. I also could not find the file library.tsv in the fragpipe output folder and it seems like the file generation error is related to the "KeyError: irt" (line 5217 in log file). Quick details: 0) I followed the guide on https://fragpipe.nesvilab.org/docs/tutorial_DIA for DIA_SpecLib_Quant and use default values for this workflow in FragPipe. 1) My DIA files in mzML format do not have overlapping windows. I am not sure what is meant with staggered windows though. 2) I used iRT peptides, so I changed the RT calibration option to iRT. 3) Ubuntu 18.04 LTS, 128GB RAM, FragPipe v17.1

I would appreciate your help very much!

Cheers, jens

fcyu commented 2 years ago

It is due to the error from EasyPQP:

Info: 26440 redundant PSMs identified after filtering with /home/dalco/awesome_proteins/fragpipe_out_debugging/psm.tsv and /home/dalco/awesome_proteins/fragpipe_out_debugging/peptide.tsv
Traceback (most recent call last):
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3080, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 4554, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 4562, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'irt'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dalco/anaconda3/bin/easypqp", line 8, in <module>
    sys.exit(cli())
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1690, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/easypqp/main.py", line 111, in library
    generate(infiles, outfile, psmtsv, peptidetsv, rt_referencefile, rt_reference_run_path, rt_filter, im_referencefile, im_reference_run_path, im_filter, psm_fdr_threshold, peptide_fdr_threshold, protein_fdr_threshold, rt_lowess_fraction, rt_psm_fdr_threshold, im_lowess_fraction, im_psm_fdr_threshold, pi0_lambda, peptide_plot_path, protein_plot_path, min_peptides, proteotypic, consensus, nofdr)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/easypqp/library.py", line 423, in generate
    pepida = pepida.loc[np.isfinite(pepida['irt'])]
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py", line 3024, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3082, in get_loc
    raise KeyError(key) from err
KeyError: 'irt'
Info: easypqp_rt_alignment_jsettelmeier_Group_A_054; Peptide overlap between run and reference: 0.

Guo Ci @guoci , can you take a look?

Thanks,

Fengchao

guoci commented 2 years ago

There are problems with alignment due to no overlap of peptides between runs, did you try ciRT? You also mentioned you used iRT, but they are appearing in your search results?

Info: easypqp_rt_alignment_jsettelmeier_Group_A_054; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_054; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_055; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_055; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_056; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_056; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_057; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_057; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_073; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_073; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_074; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_074; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_075; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_075; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_076; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_076; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_078; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_078; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_079; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_079; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_080; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_080; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_081; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_081; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_100; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_100; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_101; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_101; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_103; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_103; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_104; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_104; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_105; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_105; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_106; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_106; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_122; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_122; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_123; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_123; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_124; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_124; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_125; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_125; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_98; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_98; Skipping run because not enough peptides could be found for alignment.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_99; Peptide overlap between run and reference: 0.
Info: easypqp_rt_alignment_jsettelmeier_Group_A_99; Skipping run because not enough peptides could be found for alignment.
Library not generated, not enough peptides could be found for alignment.
Please try using other options for alignment (e.g. ciRT if used other options)
fcyu commented 2 years ago

@guoci I think the error is due to KeyError: 'irt' according the the message

Traceback (most recent call last):
  File "/home/dalco/anaconda3/bin/easypqp", line 8, in <module>
    sys.exit(cli())
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1690, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/easypqp/main.py", line 111, in library
    generate(infiles, outfile, psmtsv, peptidetsv, rt_referencefile, rt_reference_run_path, rt_filter, im_referencefile, im_reference_run_path, im_filter, psm_fdr_threshold, peptide_fdr_threshold, protein_fdr_threshold, rt_lowess_fraction, rt_psm_fdr_threshold, im_lowess_fraction, im_psm_fdr_threshold, pi0_lambda, peptide_plot_path, protein_plot_path, min_peptides, proteotypic, consensus, nofdr)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/easypqp/library.py", line 423, in generate
    pepida = pepida.loc[np.isfinite(pepida['irt'])]
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py", line 3024, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/dalco/anaconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3082, in get_loc
    raise KeyError(key) from err
KeyError: 'irt'

Best,

Fengchao

guoci commented 2 years ago

@fcyu probably not. The lines starting with Info: are printed to stdout, the traceback to stderr. The output is probably reordered in the log is due to this.

JensSettelmeier commented 2 years ago

@guoci yes, I spiked the iRTs into my samples and they were found by Spectronaut. According to the Guide to FragPipe results https://fragpipe.nesvilab.org/docs/tutorial_fragpipe_outputs.html, I would expect to find the iRTs in peptide.tsv, which is generated by Philosopher. But I couldn't find them in peptide.tsv, however there is a iRT.tsv, where the 11 iRTs with 7 charge states are listed, but not sure if it means they were found.

anesvi commented 2 years ago

Did you add iRT peptides to the database?

If yes, we need to check if something filtered them out later

Get Outlook for iOShttps://aka.ms/o0ukef


From: Jens Settelmeier @.> Sent: Monday, May 23, 2022 2:52:27 PM To: Nesvilab/FragPipe @.> Cc: Subscribed @.***> Subject: Re: [Nesvilab/FragPipe] [0:00] Loading spectral library library.tsv cannot read the file ERROR: src/diann.cpp: 17316: Cannot load spectral library Process 'DIA-NN' finished, exit code: 255 (Issue #687)

External Email - Use Caution

@guocihttps://github.com/guoci yes, I spiked the iRTs into my samples and they were found by Spectronaut. According to the Guide to FragPipe results https://fragpipe.nesvilab.org/docs/tutorial_fragpipe_outputs.html, I would expect to find the iRTs in peptide.tsv, which is generated by Philosopher. But I couldn't find them in peptide.tsv, however there is a iRT.tsv, where the 11 iRTs with 7 charge states are listed, but not sure if it means they were found.

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/687#issuecomment-1135025345, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM67FZMYT5YXXMZN523TVLPHWXANCNFSM5WVEVRUQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>


Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

guoci commented 2 years ago

@JensSettelmeier iRT.tsv is written by FragPipe, not part of the search.

JensSettelmeier commented 2 years ago

@anesvi good point, no, I did not manually add the iRTs to the database. I also could not find any of the 11 sequences via "grep" in the .fas file with the proteomes of the organisms I am searching against, which was extended with contaminants and decoys due to the "Add decoys and contaminants" function. So they are not added if the "Add decoys and contaminants" function or "RT calibration" with "iRT" option is used, right? Is this to expect? Adding the iRTs to the database should solve the problem then. I will let you know if it worked. Thank you all for your help!

JensSettelmeier commented 2 years ago

log_2022-05-23_23-02-52_spectral_library_still_missing.txt

I added the iRT peptides https://biognosys.com/content/uploads/2021/03/irtfusion.fasta to the database, but the error "cannot load spectral library" and the "keyError: 'irt' persist. Logfile is attached.

guoci commented 2 years ago

There is still no overlap, are the peptides in your search results?

fcyu commented 2 years ago

The search does not look right. In the old log (without iRT), there are

time="16:36:54" level=info msg="Database search results" ions=124693 peptides=116788 psms=461539
time="16:36:55" level=info msg="Converged to 1.00 % FDR with 16094 PSMs" decoy=162 threshold=0.884341 total=16256
time="16:36:55" level=info msg="Converged to 0.98 % FDR with 203 Peptides" decoy=2 threshold=0.978443 total=205
time="16:36:56" level=info msg="Converged to 0.92 % FDR with 324 Ions" decoy=3 threshold=0.976067 total=327
time="16:36:56" level=info msg="Protein inference results" decoy=601 target=723
time="16:36:56" level=info msg="Converged to 3.70 % FDR with 27 Proteins" decoy=1 threshold=0.9888 total=28
time="16:37:01" level=info msg="2D FDR estimation: Protein mirror image" decoy=27 target=27
time="16:37:01" level=info msg="Second filtering results" ions=795 peptides=496 psms=42503
time="16:37:01" level=info msg="Converged to 1.00 % FDR with 41708 PSMs" decoy=421 threshold=0.002434 total=42129
time="16:37:01" level=info msg="Converged to 0.74 % FDR with 269 Peptides" decoy=2 threshold=0.832473 total=271
time="16:37:01" level=info msg="Converged to 0.84 % FDR with 475 Ions" decoy=4 threshold=0.735302 total=479

In the new log, it becomes

time="22:54:50" level=info msg="Database search results" ions=14637 peptides=14206 psms=38089
time="22:54:50" level=info msg="Converged to 0.99 % FDR with 2612 PSMs" decoy=26 threshold=0.814079 total=2638
time="22:54:50" level=info msg="Converged to 0.00 % FDR with 68 Peptides" decoy=0 threshold=0.985655 total=68
time="22:54:50" level=info msg="Converged to 0.00 % FDR with 88 Ions" decoy=0 threshold=0.985655 total=88
time="22:54:50" level=info msg="Protein inference results" decoy=67 target=135
time="22:54:50" level=info msg="Converged to 9.09 % FDR with 22 Proteins" decoy=2 threshold=0.9826 total=24
time="22:54:50" level=info msg="2D FDR estimation: Protein mirror image" decoy=22 target=22
time="22:54:50" level=info msg="Second filtering results" ions=191 peptides=130 psms=4493
time="22:54:50" level=info msg="Converged to 0.80 % FDR with 4457 PSMs" decoy=36 threshold=0 total=4493
time="22:54:50" level=info msg="Converged to 0.91 % FDR with 109 Peptides" decoy=1 threshold=0.446526 total=110
time="22:54:50" level=info msg="Converged to 0.62 % FDR with 159 Ions" decoy=1 threshold=0.446526 total=160

Much fewer IDs.

Best,

Fengchao

JensSettelmeier commented 2 years ago

log_2022-05-23_23-29-09_spectral_library_tsv_missing_msfraggerDIA.txt

uh sorry for the confusion. I sent by accident the log file for issue #539! Here the correct log file.

fcyu commented 2 years ago

Thanks, after taking another look, there are only 260 peptides from 24 runs. I guess either something is wrong in your settings or there is not many IDs in your data.

Best,

Fengchao

JensSettelmeier commented 2 years ago

@guoci the iRT peptides are not in the peptide.tsv file. But the iRT peptides were found with Spectronaut.

@fcyu yes I expect a little number of peptide and protein IDs since only a handful of proteins were mixed, so this is fine.

fcyu commented 2 years ago

the iRT peptides are not in the peptide.tsv file. But the iRT peptides were found with Spectronaut.

If so, please double check your settings.

Best,

Fengchao

JensSettelmeier commented 2 years ago

log_2022-05-24_19-30-47.txt

I double checked the settings -> rerun the search -> redo and follow the instructions on https://fragpipe.nesvilab.org/docs/tutorial_DIA including to change the RT calibration option to iRT, keep all TSV files and keep intermediate files. The pipe still breaks and error persists, meaning:

1) DIA-NN can not read library.tsv 2) Can not find library.tsv in the fragpipe output folder 3) Can not find the iRT peptides in the peptide.tsv

I redid the analysis on Spectronaut, and found the iRTs in the "QC section -> iRT graph". Does FragPipe filter out the iRTs in the peptide.tsv file?

A detail I didn't mention yet: For FragPipe, I convert the raw files to mzML with the latest version of the official proteodiscover msconvert docker image (https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses) using the parameters '--64 --zlib --filter "peakPicking true 1-', which should be fine? However, when loading the mzML files in FragPipe (second instruction in the "DIA_SpecLib_Quant" tutorial), FragPipe recognizes the data as DDA, so I have to change the file type to DIA manually, which should also be fine? But why is it not recognized as DIA automatically?

Maybe also good to know: The DIA_SpecLib_Quant workflow works fine on the provided "Mouse Tutorial". If I use exactly the same settings as in the Mouse Tutorial, but replacing the mouse data with my DIA mzML files the process fails: log_2022-05-24_19-32-37.txt Note the only difference in the error logs corresponding two workflows are: using iRTs for RT calibration, keep all TSV files and keep intermediate files. Keeping files should not make any difference.

anesvi commented 2 years ago

And if you do not see iRT peptides in PSM.tsv, can you check you added iRT to fasta:

You should add the following sequence to fasta:

Biognosys|iRT-Kit_WR_fusion GN=iRTKit LGGNEQVTRYILAGVENSKGTFIIDPGGVIRGTFIIDPAAVIRGAGSSEPVTGLDAKTPVISGGPYEYRVEATFGVDESNAKTPVITGAPYEYRDGLDAASYYAPVRADVTPADFSEWSKLFLQFGAQGSPFLK

From: Jens Settelmeier @.> Sent: Tuesday, May 24, 2022 1:47 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Mention @.> Subject: Re: [Nesvilab/FragPipe] [0:00] Loading spectral library library.tsv cannot read the file ERROR: src/diann.cpp: 17316: Cannot load spectral library Process 'DIA-NN' finished, exit code: 255 (Issue #687)

External Email - Use Caution

log_2022-05-24_19-30-47.txthttps://github.com/Nesvilab/FragPipe/files/8764972/log_2022-05-24_19-30-47.txt

I double checked the settings -> rerun the search -> redo and follow the instructions on https://fragpipe.nesvilab.org/docs/tutorial_DIA including to change the RT calibration option to iRT, keep all TSV files and keep intermediate files. The pipe still breaks and error persists, meaning:

  1. DIA-NN can not read library.tsv
  2. Can not find library.tsv in the fragpipe output folder
  3. Can not find the iRT peptides in the peptide.tsv

I redid the analysis on Spectronaut, and found the iRTs in the "QC section -> iRT graph". Does FragPipe filter out the iRTs in the peptide.tsv file?

A detail I didn't mention yet: For FragPipe, I convert the raw files to mzML with the latest version of the official proteodiscover msconvert docker image (https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses) using the parameters '--64 --zlib --filter "peakPicking true 1-', which should be fine? However, when loading the mzML files in FragPipe (second instruction in the "DIA_SpecLib_Quant" tutorial), FragPipe recognizes the data as DDA, so I have to change the file type to DIA manually, which should also be fine? But why is it not recognized as DIA automatically?

Maybe also good to know: The DIA_SpecLib_Quant workflow works fine on the provided "Mouse Tutorial". If I use exactly the same settings as in the Mouse Tutorial, but replacing the mouse data with my DIA mzML files the process fails: log_2022-05-24_19-32-37.txthttps://github.com/Nesvilab/FragPipe/files/8764984/log_2022-05-24_19-32-37.txt Note the only difference in the error logs corresponding two workflows are: using iRTs for RT calibration, keep all TSV files and keep intermediate files. Keeping files should not make any difference.

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/687#issuecomment-1136256477, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM62UAPGOUWCRCRXJH73VLUIX3ANCNFSM5WVEVRUQ. You are receiving this because you were mentioned.Message ID: @.**@.>>


Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

JensSettelmeier commented 2 years ago

The iRTs are also not included in the PSM.tsv file and I included the iRTs by adding exactly the mentioned sequence to the fasta.

anesvi commented 2 years ago

Well, I see them in my PSM.tsv in my data I tried. Need to see your data. Send me 1-2 files. I cannot think of anything now without testing with your data and your fasta database (please pass to us as well)

From: Jens Settelmeier @.> Sent: Tuesday, May 24, 2022 2:17 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Mention @.> Subject: Re: [Nesvilab/FragPipe] [0:00] Loading spectral library library.tsv cannot read the file ERROR: src/diann.cpp: 17316: Cannot load spectral library Process 'DIA-NN' finished, exit code: 255 (Issue #687)

External Email - Use Caution

The iRTs are also not included in the PSM.tsv file and I included the iRTs by adding exactly the mentioned sequence to the fasta.

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/687#issuecomment-1136285627, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM66HCTBVYFNEZANOT23VLUMI5ANCNFSM5WVEVRUQ. You are receiving this because you were mentioned.Message ID: @.**@.>>


Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

JensSettelmeier commented 2 years ago

log_2022-05-24_20-21-31.txt I also run the workflow "DIA_SpecLib_Quant" mouse example again, but only using the DIA files. Since iRTs are included in the sample (according to the paper), I checked the option "RT calibration" with "iRT". The workflow crashes with the same error message as it does with my data (attached log). Is this to expect? If not, it would be nice if you could verify the error.

anesvi commented 2 years ago

No, crashed are not expected.

Can you send me a link to these files you tried, if you cannot share your own data

From: Jens Settelmeier @.> Sent: Tuesday, May 24, 2022 2:28 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Mention @.> Subject: Re: [Nesvilab/FragPipe] [0:00] Loading spectral library library.tsv cannot read the file ERROR: src/diann.cpp: 17316: Cannot load spectral library Process 'DIA-NN' finished, exit code: 255 (Issue #687)

External Email - Use Caution

log_2022-05-24_20-21-31.txthttps://github.com/Nesvilab/FragPipe/files/8765305/log_2022-05-24_20-21-31.txt I also run the workflow "DIA_SpecLib_Quant" mouse example again, but only using the DIA files. Since iRTs are included in the sample (according to the paperhttps://pubs.acs.org/doi/pdf/10.1021/acs.jproteome.8b00898), I checked the option "RT calibration" with "iRT". The workflow crashes with the same error message as it does with my data (attached log). Is this to expect? If not, it would be nice if you could verify the error.

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/687#issuecomment-1136295969, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM667R4D5HSB4GVNOL2LVLUNRVANCNFSM5WVEVRUQ. You are receiving this because you were mentioned.Message ID: @.**@.>>


Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

JensSettelmeier commented 2 years ago

Thanks a lot! I sent you a mail with two links: one to my data and one to the speclib-raw data.

Cheers, Jens

fcyu commented 2 years ago

Problem solved after contact by email.

cpanse commented 1 year ago

Problem solved after contact by email.

@fcyu @JensSettelmeier Would love to know the solution. I ran into the same issue while using Linux cmd.

fcyu commented 1 year ago

Hi @cpanse ,

I think @JensSettelmeier 's issue has been fixed in the updated tool. And that issue seems to be data specific. Could you send us your log file?

Thanks,

Fengchao

cpanse commented 1 year ago

Hi @cpanse ,

I think @JensSettelmeier 's issue has been fixed in the updated tool. And that issue seems to be data specific. Could you send us your log file?

Thanks,

Fengchao

@fcyu read the other issues, meanwhile. In my docker-image, I have to make fragpipe understand where easypqp is installed without running the X windows. C

 /usr/local/nesvilab/fragpipe/bin/fragpipe --headless --workflow /scratch/FRAGPIPEDIA_312/o29661/FragPipe_workflow.workflow --manifest /scratch/FRAGPIPEDIA_312/o29661/manifest.fp-manifest --workdir /scratch/FRAGPIPEDIA_312/o29661 --config-python /usr/bin/python3 --config-msfragger /usr/local/nesvilab/MSFragger-3.5/MSFragger-3.5.jar --config-philosopher /usr/local/nesvilab/philosopher_v4.4.0 --ram 128
I have no name!@57fd57211b2a:/scratch/FRAGPIPEDIA_312/o29661$ which easypqp
/usr/local/bin/easypqp
fcyu commented 1 year ago

Hi @cpanse ,

Do you have that you have to specify --config-python every time you run FragPipe? If so, can you see what the fragpipe-config.bin-python is in the fragpipe-ui.cache file? The cache file is normally in ~/.config/FragPipe/fragpipe if you didn't change the XDG_CONFIG_HOME environment variable.

Best,

Fengchao

cpanse commented 1 year ago

Hmm; I have never run fragpipe-ui in my life. So I need to find a template. I will continue tomorrow. Thank you for your instant response. I'll let you know if that works. Thanks again, and best wishes from Zurich, Christian.

On Tue, Nov 15, 2022 at 5:22 PM Fengchao @.***> wrote:

Hi @cpanse https://github.com/cpanse ,

Do you have that you have to specify --config-python every time you run FragPipe? If so, can you see what the fragpipe-config.bin-python is in the fragpipe-ui.cache file? The cache file is normally in ~/.config/FragPipe/fragpipe if you didn't change the XDG_CONFIG_HOME environment variable.

Best,

Fengchao

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/FragPipe/issues/687#issuecomment-1315553338, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFMYYZGUDJL62N4PP2HFQ3WIO2EJANCNFSM5WVEVRUQ . You are receiving this because you were mentioned.Message ID: @.***>

fcyu commented 1 year ago

You don't need to create fragpipe-ui.cache file by yourself. FragPipe will generate and overwrite since it is a cache file.

Best,

Fengchao

cpanse commented 1 year ago

Dear Fengchao,

I used Xnest and some X-Windows magic to connect to the compute node's docker instance and run FragPipe GUI. Luckily the FragPipe-GUI (v18) told me that the Cython package is unavailable.

Now I get the missing cmd line in the cmd run. /usr/bin/python3 -u /usr/local/nesvilab/fragpipe/tools/speclib/gen_con_spec_lib.py ...

for whatsoever reason I add my Dockerfile below

thanks again and best wishes, C

FROM debian:11

# 2021/2022
MAINTAINER Christian Panse ***@***.***>

RUN apt-get update && apt-get install -y --no-install-recommends apt-utils
unzip less openjdk-17-jre openjdk-17-jdk libgomp1 python3 pip git vim
RUN mkdir -p /usr/local/nesvilab/ /usr/local/nesvilab/philosopher_v4.5.1/

# required input; see ***@***.***:~/FragPipe-Docker
COPY FragPipe-18.0.zip /usr/local/nesvilab/FragPipe-18.0.zip
COPY philosopher-v4.5.1-RC5
/usr/local/nesvilab/philosopher_v4.5.1/philosopher-v4.5.1-RC5
COPY philosopher_v4.4.0 /usr/local/nesvilab/
COPY MSFragger-3.5.zip /usr/local/nesvilab/

RUN cd /usr/local/nesvilab/ && unzip FragPipe-18.0.zip
RUN cd /usr/local/nesvilab/ && unzip MSFragger-3.5.zip

# remember from ISB software
RUN ln -sfv /bin/mv /usr/local/bin/mv

ENV JAVA_HOME="/usr/lib/jvm/java-17-openjdk-amd64/"

# 2022-11-15 the Cython package was missing
RUN pip install lxml Cython

# TODO(cp): better use a release
RUN pip install ***@***.***

# # consider; there is an \r but no \n
# RUN sed -i "1s/.*/\#\!\/usr\/bin\/env python3\n/"
/usr/local/nesvilab/fragpipe/tools/speclib/gen_con_spec_lib.py
RUN mkdir /usr/local/nesvilab/fragpipe/cache && chmod 777
/usr/local/nesvilab/fragpipe/cache

# for debug; just to have X Window System available
RUN apt-get install xterm -y

On Wed, Nov 16, 2022 at 1:02 AM Fengchao @.***> wrote:

You don't need to create fragpipe-ui.cache file by yourself. FragPipe will generate and overwrite since it is a cache file.

Best,

Fengchao

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/FragPipe/issues/687#issuecomment-1316056177, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFMYYYOLH2OOXNP75YHRJDWIQQA3ANCNFSM5WVEVRUQ . You are receiving this because you were mentioned.Message ID: @.***>