singularity pull issue

[HFan11]

Hi,

I want to use it in the singularity containers but when I use the demand like:

singularity pull docker://pyprophet/pyprophet:2.1.3

It is not working. Do you know the correct code like using executables or others?

[singjc]

Hi,

I don't think you are using the correct command for creating a singularity contained from a docker image. In your command, you left out the output file for saving the container to

Have you tried:

singularity pull pyprophet.sif docker://pyprophet/pyprophet:2.1.3

See: singularity pull examples

[HFan11]

Sorry for misunderstanding. I tried your command. Also, in my previous command I use the remove command to my folder. in my script to merge the .osw file, it shows the bug that:

exec /usr/local/bin/pyprophet failed: a shared library is likely missing in the image

However, when I checked it, it shows that:

Singularity> ls /usr/local/bin
2to3          f2py3          pip3       python3-config
2to3-3.7      f2py3.7        pip3.7     python3.7
cygdb         futurize       pydoc      python3.7-config
cython        hyperopt-mongo-worker  pydoc3     python3.7m
cythonize     idle           pydoc3.7   python3.7m-config
easy_install      idle3          pyprophet  pyvenv
easy_install-3.6  idle3.7        python     pyvenv-3.7
easy_install-3.7  pasteurize         python-config  tqdm
f2py          pip            python3    wheel
``` Does this image fit to the singularity container?

[singjc]

Did the singularity command above create the pyprophet.sif image without any errors?

Do you see this:

$ singularity pull pyprophet.sif docker://pyprophet/pyprophet:2.1.3
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob e79bb959ec00 done
...
2024/06/27 12:28:48  info unpack layer: sha256:a7739eadfc518bb9f9a23250bedb85864fb1e36306e5ce2a1e8c3ef04e0ca659
INFO:    Creating SIF file...

check to see if it created the image

$ ls -lh *sif
-rwxrwxr-x 1 singjc singjc 456M Jun 27 12:29 pyprophet.sif

Check to see if the image works correctly

$ singularity exec pyprophet.sif pyprophet --help
Usage: pyprophet [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

  PyProphet: Semi-supervised learning and scoring of OpenSWATH results.

  Visit http://openswath.org for usage instructions and help.

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  backpropagate    Backpropagate multi-run peptide and protein scores to...
  export           Export TSV/CSV tables
  export-compound  Export Compound TSV/CSV tables
  filter           Filter sqMass files
  ipf              Infer peptidoforms after scoring of MS1, MS2 and...
  merge            Merge multiple OSW files and (for large experiments, it...
  peptide          Infer peptides and conduct error-rate estimation in...
  protein          Infer proteins and conduct error-rate estimation in...
  reduce           Reduce scored PyProphet file to minimum for global...
  score            Conduct semi-supervised learning and error-rate...
  subsample        Subsample OpenSWATH file to minimum for integrated...

[HFan11]

yes, i am sure the image works correctly as you showed. However, it still shows that:

exec /usr/local/bin/pyprophet failed: a shared library is likely missing in the image

it happens at my first script line:

# Merge OSW files
singularity exec -B /labs/mpsnyder/hfan11/output:/output -B /labs/mpsnyder/hfan11/lib:/lib -B /labs/mpsnyder/hfan11/merged:/merged /labs/mpsnyder/hfan11/pyprophet.sif pyprophet merge --template=/lib/SpyogenesAssayLibrary_decoy.pqp --out=/merged/merged.osw /output/*.osw

[singjc]

Strange. I cannot reproduce the error you are getting.

development$ singularity exec -B /home/singjc/development/tmp/:/tmp/ pyprophet.sif pyprophet merge --template=/tmp/tmp.osw --out=/tmp/merged.osw /tmp/tmp.osw /tmp/tmp2.osw
Info: Merged runs of file /tmp/tmp.osw to /tmp/merged.osw.
Info: Merged runs of file /tmp/tmp2.osw to /tmp/merged.osw.
Info: Merged generic features of file /tmp/tmp.osw to /tmp/merged.osw.
Info: Merged generic features of file /tmp/tmp2.osw to /tmp/merged.osw.
Info: Merged MS1 features of file /tmp/tmp.osw to /tmp/merged.osw.
Info: Merged MS1 features of file /tmp/tmp2.osw to /tmp/merged.osw.
Info: Merged MS2 features of file /tmp/tmp.osw to /tmp/merged.osw.
Info: Merged MS2 features of file /tmp/tmp2.osw to /tmp/merged.osw.
Info: Merged transition features of file /tmp/tmp.osw to /tmp/merged.osw.
Info: Merged transition features of file /tmp/tmp2.osw to /tmp/merged.osw.
Info: All OSWS files were merged.
development$ ls -ltrh tmp/
total 606M
-rw-rw-r-- 1 singjc singjc 165M Jul  1 23:20 tmp.osw
-rw-rw-r-- 1 singjc singjc 165M Jul  1 23:47 tmp2.osw
-rw-rw-r-- 1 singjc singjc 277M Jul  1 23:47 merged.osw

Are you running your script on a cluster? If so, does the image work both locally and remotely on the cluster?

Can you copy and paste the output of singularity exec pyprophet.sif pyprophet merge --help so that we can verify that the image does work as expected. If you still get the error, then there is something wrong with the image. Which means there is some error that occurred during the building of the singularity image.

I can alternatively, send you a copy of the pyprophet.sif image (453Mb) that I have used above to test if that works, please just provide me with an email address or place to share it.

[HFan11]

The output is:

$ singularity exec pyprophet.sif pyprophet merge --help
INFO:    Converting SIF file to temporary sandbox...
Usage: pyprophet merge [OPTIONS] [INFILES]...

  Merge multiple OSW files and (for large experiments, it is recommended to
  subsample first).

Options:
  --out PATH                  Merged OSW output file.  [required]
  --same_run / --no-same_run  Assume input files are from same run (deletes
                              run information).
  --template PATH             Template OSW file.  [required]
  --help                      Show this message and exit.
INFO:    Cleaning up image...

Yes I ran it on the cluster. Does it work as expected? I still get the error today after reinstalling several times. My email address is hfan11@stanford.edu. Thanks for sharing!