search

Nesvilab / philosopher

PeptideProphet, PTMProphet, ProteinProphet, iProphet, Abacus, and FDR filtering
https://philosopher.nesvilab.org
GNU General Public License v3.0
111 stars 19 forks source link

Philosopher docker issue #192

Closed tsigalko2003 closed 3 years ago

tsigalko2003 commented 3 years ago

Describe the bug identical job ( both input and command arguments) give error message " sh: 1: mv: not found sh: 1: mv: not found sh: 1: mv: not found"

To Reproduce Steps to reproduce the behavior:

  1. create the philosopher docker image based on Ubuntu:18.04 and install philosopher( We had tried both compiling with go and the binary file, but returning the same issue).

  2. within the docker container, run command philosopher peptideprophet --nonparam --expectscore --decoyprobs --ppm --accmass --decoy rev_ --database /data/docker/proteinref.fas ./*.pepXML the output is INFO[22:34:36] Executing PeptideProphet v3.4.13 file 1: /data/docker/JD_06232014_sample1_A.pepXML processed altogether 35125 results INFO: Results written to file: /data/docker/interact-JD_06232014_sample1_A.pep.xml

    • /data/docker/interact-JD_06232014_sample1_A.pep.xml
    • Building Commentz-Walter keyword tree...
    • Searching the tree...
    • Linking duplicate entries...
    • Printing results...

sh: 1: mv: not found sh: 1: mv: not found sh: 1: mv: not found failed to update /data//docker/interact-JD_06232014_sample1_A.pep.xml from temp file /data/docker/interact-JD_06232014_sample1_A.pep.xml.tmp.3rRKuK, error 0 "Success"failed to update /data/docker/interact-JD_06232014_sample1_A.pep.xml from temp file /data/docker/interact-JD_06232014_sample1_A.pep.xml.tmp.3rRKuK, error 0 "Success"failed to update /data/docker/interact-JD_06232014_sample1_A.pep.xml from temp file /data/docker/interact-JD_06232014_sample1_A.pep.xml.tmp.3rRKuK, error 0 "Success"error: no RefreshParser data written to file /data/docker/interact-JD_06232014_sample1A.pep.xml using Accurate Mass Bins using PPM mass difference Using Decoy Label "rev". Decoy Probabilities will be reported. Using non-parametric distributions (X! Tandem) (using Tandem's expectation score for modeling) adding ACCMASS mixture distribution using search_offsets in ACCMASS mixture distr: 0 init with X! Tandem stricttrypsin MS Instrument info: Manufacturer: UNKNOWN, Model: UNKNOWN, Ionization: UNKNOWN, Analyzer: UNKNOWN, Detector: UNKNOWN

INFO: Processing standard MixtureModel ... PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906251008-exported (Linux-x86_64)) AKeller@ISB read in 0 1+, 25673 2+, 8811 3+, 606 4+, 19 5+, 16 6+, and 0 7+ spectra. Initialising statistical models ... Found 5236 Decoys, and 29889 Non-Decoys Iterations: .........10.........20...... WARNING: Mixture model quality test failed for charge (1+). WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 27 iterations sh: 1: mv: not found sh: 1: mv: not found sh: 1: mv: not found failed to update /data/docker/interact-JD_06232014_sample1_A.pep.xml from temp file /data/docker/interact-JD_06232014_sample1_A.pep.xml.tmp.Xih7OJ, error 0 "Success"failed to update /data/docker/interact-JD_06232014_sample1_A.pep.xml from temp file /data/docker/interact-JD_06232014_sample1_A.pep.xml.tmp.Xih7OJ, error 0 "Success"failed to update /data/docker/interact-JD_06232014_sample1_A.pep.xml from temp file data/docker/interact-JD_06232014_sample1_A.pep.xml.tmp.Xih7OJ, error 0 "Success"error: no peptideprophet data written to file /data/docker/interact-JD_06232014_sample1_A.pep.xml INFO[22:35:54] Done

  1. We tried run the same command at the same location in the host, rather than the docker with identical binary file , everything is working without any mv-not-found error

Any help will be appreciated

prvst commented 3 years ago

Hi, @tsigalko2003. I'll try to reproduce your issue here.

theoplatt commented 3 years ago

@prvst I know a "me too" doesn't help solve the problem but we're seeing the same issue with Docker as @tsigalko2003

Thanks!

prvst commented 3 years ago

Acknowledged, and thanks for the feedback. I'm planning to release a new version soon with some nice features. The release will include a new container. Until then, what you can do is download the binary and build an image locally. Since Philosopher is self-contained and has zero dependencies, the only directive you need to add to your Dockerfile is the ADD. Below is the current recipe we're using:

FROM biocontainers/biocontainers:latest

LABEL base.image="biocontainers:latest"
LABEL version="1"
LABEL software="Philosopher"
LABEL description="A complete toolkit for shotgun proteomics data analysis"
LABEL website="https://philosopher.nesvilab.org"
LABEL documentation="https://philosopher/wiki"
LABEL license="GPL-3.0"
LABEL BIOTOOLS=""
LABEL tags="Proteomics"

# MAINTAINER Felipe da Veiga Leprevost <felipevl@umich.edu>

USER biodocker

ADD philosopher /home/biodocker/bin/

ENV PATH /home/biodocker/bin/:$PATH

WORKDIR /data/
theoplatt commented 3 years ago

There must be some dependency in the biocontainers/biocontainers image that is missing from ubuntu:18.04 because if I just replace the FROM statement and add the user creation I get the same weird 'mv: not found' error. Some Go dependency maybe?

FROM ubuntu:18.04
RUN adduser biodocker
prvst commented 3 years ago

Hi @theoplatt. You are right. I think that the biocontainers image must need some updates. I got it working by changing the image, so I'll update the core image with the latest Ubuntu image. Thanks for the update.

prvst commented 3 years ago

Fixed. Added to v3.6.0

theoplatt commented 3 years ago

Hi @prvst - thanks for all your work on this project. I still get the same issue with the very latest container. Is there something in the way you determine linux vs. windows that is making it fall over? Is it somehow changing the path?

Here are the steps to recreate. I can share the fasta and .pepXML wile with you but they are not special. I just use the volume mounting as a way to get the data into the container.

Added blank lines between the commands for readability

(philosopher) Theos-MacBook-Pro-2:philosopher-container theoplatt$ docker run -v /Users/theoplatt/git/philosopher-container:/mydata  -it  prvst/philosopher:4.0.0
root@b696e1112d6a:/data# cp /mydata/UP000005640_9606.target.decoy.con.20200820.fasta .
root@b696e1112d6a:/data# cp /mydata/123.pepXML .
root@b696e1112d6a:/data# ls
123.pepXML  UP000005640_9606.target.decoy.con.20200820.fasta

root@b696e1112d6a:/data# philosopher workspace --init
INFO[17:59:43] Executing Workspace  v4.0.0                  
WARN[17:59:43] can't check for updates, server unreachable  
INFO[17:59:43] Creating workspace                           
INFO[17:59:43] Done                                         

root@b696e1112d6a:/data# philosopher database --annotate UP000005640_9606.target.decoy.con.20200820.fasta
INFO[17:59:52] Executing Database  v4.0.0                   
INFO[17:59:52] Annotating the database                      
INFO[17:59:57] Done                            

root@b696e1112d6a:/data# philosopher peptideprophet --database UP000005640_9606.target.decoy.con.20200820.fasta --decoy decoy_ --ppm --accmass --expectscore --decoyprobs --nonparam 123.pepXML
INFO[18:03:15] Executing PeptideProphet  v4.0.0             
 file 1: /data/123.pepXML
Unknown file type. No file loaded./data/123.d
WARNING: cannot open data file /data/123.d in msms_run_summary tag... unrecognized extension .d, trying .mzML ...
WARNING: CANNOT correct data file /data/123.mzML in msms_run_summary tag...
Unknown file type. No file loaded./data/123.d
WARNING: cannot open data file /data/123.d in msms_run_summary tag... unrecognized extension .d, trying .mzML ...
WARNING: CANNOT correct data file /data/123.mzML in msms_run_summary tag...
 processed altogether 68996 results
INFO: Results written to file: /data/interact-123.pep.xml

  - /data/interact-123.pep.xml
  - Building Commentz-Walter keyword tree...
  - Searching the tree...
  - Linking duplicate entries...
  - Printing results...

sh: 1: mv: not found
sh: 1: mv: not found
sh: 1: mv: not found
failed to update /data/interact-123.pep.xml from temp file /data/interact-123.pep.xml.tmp.LguNgW, error 0 "Success"failed to update /data/interact-123.pep.xml from temp file /data/interact-123.pep.xml.tmp.LguNgW, error 0 "Success"failed to update /data/interact-123.pep.xml from temp file /data/interact-123.pep.xml.tmp.LguNgW, error 0 "Success"error: no RefreshParser data written to file /data/interact-123.pep.xml
using Accurate Mass Bins
using PPM mass difference
Using Decoy Label "decoy_".
Decoy Probabilities will be reported.
Using non-parametric distributions
 (X! Tandem) (using Tandem's expectation score for modeling)
adding ACCMASS mixture distribution
using search_offsets in ACCMASS mixture distr: 0
init with X! Tandem stricttrypsin 
MS Instrument info: Manufacturer: UNKNOWN, Model: UNKNOWN, Ionization: UNKNOWN, Analyzer: UNKNOWN, Detector: UNKNOWN

INFO: Processing standard MixtureModel ... 
 PeptideProphet  (TPP v5.2.1-dev Flammagenitus, Build 201906251008-exported (Linux-x86_64)) AKeller@ISB
 read in 671 1+, 41263 2+, 19579 3+, 5863 4+, 1620 5+, 0 6+, and 0 7+ spectra.
Initialising statistical models ...
Found 20293 Decoys, and 48703 Non-Decoys
Iterations: .........10.........20......
WARNING: Mixture model quality test failed for charge (6+).
WARNING: Mixture model quality test failed for charge (7+).
model complete after 27 iterations
sh: 1: mv: not found
sh: 1: mv: not found
sh: 1: mv: not found
failed to update /data/interact-123.pep.xml from temp file /data/interact-123.pep.xml.tmp.ejeSk3, error 0 "Success"failed to update /data/interact-123.pep.xml from temp file /data/interact-123.pep.xml.tmp.ejeSk3, error 0 "Success"failed to update /data/interact-123.pep.xml from temp file /data/interact-123.pep.xml.tmp.ejeSk3, error 0 "Success"error: no peptideprophet data written to file /data/interact-123.pep.xml
INFO[18:04:59] Done                                         
root@b696e1112d6a:/data#

Thank you!

weiranheng commented 3 years ago

Me too ,encountering same issue, please help !

weiranheng commented 3 years ago

Hello @theoplatt, I just found philosopher version 3.3.12 works well

prvst commented 3 years ago

@weiranheng are you referring to Java?

bltsai-UCLA commented 3 years ago

I am also experiencing this issue with v4.0.0:

philosopher peptideprophet --decoyprobs --ppm --accmass --nonparam --expectscore --decoy rev_ --database /data/database/2021-05-25-decoys-contam-uniprot-swissprot_canonical-isoform_human_mouse.fasta.fas /data/Exp1/1667CL_S2_Exp1_Fr1_i40_LeeJones.pepXML

INFO[23:56:50] Executing PeptideProphet v4.0.0 file 1: /data/Exp1/1667CL_S2_Exp1_Fr1_i40_LeeJones.pepXML WARNING: Unable to open file: /data/Exp1/1667CL_S2_Exp1_Fr1_i40_LeeJones.mzML cannot correct scan numbers! WARNING: Unable to open file: /data/Exp1/1667CL_S2_Exp1_Fr1_i40_LeeJones.mzML cannot correct scan numbers! processed altogether 43294 results INFO: Results written to file: /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml

sh: 1: mv: not found sh: 1: mv: not found sh: 1: mv: not found failed to update /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml from temp file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml.tmp.I6vVFr, error 0 "Success"failed to update /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml from temp file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml.tmp.I6vVFr, error 0 "Success"failed to update /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml from temp file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml.tmp.I6vVFr, error 0 "Success"error: no RefreshParser data written to file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml using Accurate Mass Bins using PPM mass difference Using Decoy Label "rev_". Decoy Probabilities will be reported. Using non-parametric distributions (X! Tandem) (using Tandem's expectation score for modeling) adding ACCMASS mixture distribution using search_offsets in ACCMASS mixture distr: 0 init with X! Tandem stricttrypsin MS Instrument info: Manufacturer: UNKNOWN, Model: UNKNOWN, Ionization: UNKNOWN, Analyzer: UNKNOWN, Detector: UNKNOWN

INFO: Processing standard MixtureModel ... PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906251008-exported (Linux-x86_64)) AKeller@ISB read in 0 1+, 19408 2+, 18484 3+, 4420 4+, 765 5+, 192 6+, and 25 7+ spectra. Initialising statistical models ... Found 15327 Decoys, and 27967 Non-Decoys Iterations: .........10.........20....... WARNING: Mixture model quality test failed for charge (1+). WARNING: Mixture model quality test failed for charge (7+). model complete after 28 iterations sh: 1: mv: not found sh: 1: mv: not found sh: 1: mv: not found failed to update /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml from temp file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml.tmp.INCZIl, error 0 "Success"failed to update /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml from temp file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml.tmp.INCZIl, error 0 "Success"failed to update /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml from temp file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml.tmp.INCZIl, error 0 "Success"error: no peptideprophet data written to file /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml INFO[23:57:59] Done

philosopher proteinprophet --maxppmdiff 2000000 --minprob 0.9 --output combined /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml /data/Exp1/interact-1667CL_S2_Exp1_Fr2_i40_LeeJones.pep.xml /data/Exp2/interact-1667CL_S3_Exp2_Fr1_i40_LeeJones.pep.xml /data/Exp2/interact-1667CL_S3_Exp2_Fr2_i40_LeeJones.pep.xml

INFO[00:14:12] Executing ProteinProphet v4.0.0 ProteinProphet (C++) by Insilicos LLC and LabKey Software, after the original Perl by A. Keller (TPP v6.0.0-rc15 Noctilucent, Build 202105021430-exported (Linux-x86_64)) (no FPKM) (using degen pep info) Reading in /data/Exp1/interact-1667CL_S2_Exp1_Fr1_i40_LeeJones.pep.xml... did not find any PeptideProphet results in input data! Did you forget to run PeptideProphet? ...read in 0 1+, 0 2+, 0 3+, 0 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.9

Reading in /data/Exp1/interact-1667CL_S2_Exp1_Fr2_i40_LeeJones.pep.xml... did not find any PeptideProphet results in input data! Did you forget to run PeptideProphet? ...read in 0 1+, 0 2+, 0 3+, 0 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.9

Reading in /data/Exp2/interact-1667CL_S3_Exp2_Fr1_i40_LeeJones.pep.xml... did not find any PeptideProphet results in input data! Did you forget to run PeptideProphet? ...read in 0 1+, 0 2+, 0 3+, 0 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.9

Reading in /data/Exp2/interact-1667CL_S3_Exp2_Fr2_i40_LeeJones.pep.xml... did not find any PeptideProphet results in input data! Did you forget to run PeptideProphet? ...read in 0 1+, 0 2+, 0 3+, 0 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.9

WARNING: no data - output file will be empty FATA[00:14:13] Cannot execute program. there was an error with ProteinProphet, please check your parameters and input files

prvst commented 3 years ago

Hey everyone. I decided to rollback our base image to ~biodocker~ biocontainers. I'm not sure why base Ubuntu is not working with our current version, so please allow me a few days to push a new container to DockerHub. Thanks for all the feedback.

theoplatt commented 3 years ago

Thanks Felipe. My guess is it's something to do with the PATH not being set correctly (or reset somehow) when the application is run. Some new Go version, or dependency in Ubuntu?

prvst commented 3 years ago

The GO code has zero dependencies, the program is statically linked. The Prophets, however (a third-party C++ application), might still do some system references. That message you see in the log comes from PeptideProphet.

weiranheng commented 3 years ago

@weiranheng are you referring to Java?

No ,I meant philosopher 3.3.12 version,I got no "sh: 1: mv: not found" error using philosopher 3.3.12 docker image.

prvst commented 3 years ago

The philosopher image was just updated with the biocontainers base image. Sorry for the wait. Please keep an eye on the release page because we are close to release an update. Thanks for reporting.

https://hub.docker.com/r/prvst/philosopher/tags