search

kentsisresearchgroup / UltraQuant

MaxQuant with snakemake and singularity workflow for open and scalable mass spectrometry data analysis on Linux computing clusters
26 stars 3 forks source link

First steps in using UltraQuant #2

Open iquasere opened 5 years ago

iquasere commented 5 years ago

Greetings

I have started trying to use UltraQuant, but since I have never worked with snakemake, and barely with mono and docker, I have encountered some problems that I haven't been able to solve.

First, it seems that the example command of snakemake makes use of arguments no longer available.

snakemake --snakefile UltraQuant.sm --configfile config.yaml --cluster "bsub -J 'myjob.{params.J}' -n {params.n} -R {params.R} -W 16:00 -o 'myjob.{params.o}' -eo 'myjob.{params.eo}'" --jn {rulename}.{jobid}.sj -j 50 -k --latency-wait 60 --use-singularity --singularity-args "--bind /data:/data,/lila:/lila,/scratch:/scratch" --ri
usage: snakemake [-h] [--snakefile FILE] [--gui [PORT]] [--cores [N]]
                 [--local-cores N] [--resources [NAME=INT [NAME=INT ...]]]
                 [--config [KEY=VALUE [KEY=VALUE ...]]] [--configfile FILE]
                 [--list] [--list-target-rules] [--directory DIR] [--dryrun]
                 [--printshellcmds] [--dag] [--rulegraph] [--d3dag]
                 [--summary] [--detailed-summary] [--touch] [--keep-going]
                 [--force] [--forceall] [--forcerun TARGET [TARGET ...]]
                 [--prioritize TARGET [TARGET ...]]
                 [--until TARGET [TARGET ...]]
                 [--omit-from TARGET [TARGET ...]] [--allow-ambiguity]
                 [--cluster CMD | --cluster-sync CMD | --drmaa [ARGS]]
                 [--cluster-config FILE] [--immediate-submit]
                 [--jobscript SCRIPT] [--jobname NAME] [--reason]
                 [--stats FILE] [--nocolor] [--quiet] [--nolock] [--unlock]
                 [--cleanup-metadata [FILE [FILE ...]]] [--rerun-incomplete]
                 [--ignore-incomplete] [--list-version-changes]
                 [--list-code-changes] [--list-input-changes]
                 [--list-params-changes] [--latency-wait SECONDS]
                 [--wait-for-files [FILE [FILE ...]]] [--benchmark-repeats N]
                 [--notemp] [--keep-target-files] [--keep-shadow]
                 [--allowed-rules ALLOWED_RULES [ALLOWED_RULES ...]]
                 [--timestamp] [--greediness GREEDINESS] [--no-hooks]
                 [--print-compilation]
                 [--overwrite-shellcmd OVERWRITE_SHELLCMD] [--verbose]
                 [--debug] [--profile FILE] [--bash-completion] [--version]
                 [target [target ...]]
snakemake: error: unrecognized arguments: --use-singularity --singularity-args

After removing such arguments, it is capable of proceeding to run the snakemake script, but fails.

SyntaxError in line 3 of ~/UltraQuant/UltraQuant.sm:
invalid syntax

This line refers to

singularity: "docker://mono:5.12.0.226"

which I promptly changed to

rule NAME:
        singularity: "docker://mono:5.12.0.226"

Still, it fails.

SyntaxError in line 4 of ~/UltraQuant/UltraQuant.sm:
Unexpected keyword singularity in rule definition (UltraQuant.sm, line 4)

My question is if you have encountered this problems, and if so is there a new version of this tool? Because it looks like a slick tool.

pillepalle123 commented 5 years ago

I would also be very interested in getting UltraQuant running. One of the issues with me is also mono. Does snakemake require a root installation of mono? I have a local installation of mono and can start MaxQuant in my mono/bin folder with: ./mono ~/MaxQuant/bin/MaxQuantCmd.exe mqpar.xml

with UltraQuant I get /bin/bash: mono: command not found

iquasere commented 5 years ago

It does seem to require such root installation, since it is pointing to /bin/bash, you need a mono executable at /usr/bin. Maybe try

alias mono='/path/to/mono'

before running UltraQuant, which should work if the command constructor in UltraQuant doesn't explicitly contain the full path of mono.

pillepalle123 commented 5 years ago

thank you for your help! that worked, I added following to my .bashrc : export PATH="/myapps/mono/bin/:$PATH"

now I am having an issue with the singularity command. I have asked our admin to install it globally since this comman wasn't found either. Now I get following error somewhere in my local Conda installation. Do you have an idea why this is? Was it actually necessary to install singularity or is it included in snakemake? Then I propably have to specify the path as well? thank you!

/myapps/UltraQuant> snakemake --snakefile UltraQuant.sm --configfile config.yaml --cluster "qsub -j y -N 'myjob.{params.J}' -pe openmpi {params.n} -l h_rt=604800" --jn {rulename}.{jobid}.sj -j 50 -k --latency-wait 60 --use-singularity Building DAG of jobs... Traceback (most recent call last): File "/myapps/Conda/lib/python3.6/site-packages/snakemake/init.py", line 544, in snakemake export_cwl=export_cwl) File "/myapps/Conda/lib/python3.6/site-packages/snakemake/workflow.py", line 566, in execute quiet=list_conda_envs) File "/myapps/Conda/lib/python3.6/site-packages/snakemake/dag.py", line 201, in pull_singularity_imgs img = singularity.Image(img_url, self) File "myapps/Conda/lib/python3.6/site-packages/snakemake/singularity.py", line 31, in init if not LooseVersion(v) >= LooseVersion("2.4.1"): File "/myapps/Conda/lib/python3.6/distutils/version.py", line 70, in ge c = self._cmp(other) File /myapps/Conda/lib/python3.6/distutils/version.py", line 337, in _cmp if self.version < other.version: TypeError: '<' not supported between instances of 'str' and 'int'

iquasere commented 5 years ago

This is very odd, since the mistake you are getting concerns the snakemake files themselves. The mistake seems simple, since it is comparing the values of "self.version" and "other.version", one of which is a string and the other is an integer. This leads me to think snakemake is not properly installed, and should be reinstalled. First, try

conda install -c bioconda snakemake

and if it claims to have already installed snakemake, first explicitly remove it and then install it.

conda remove snakemake
conda install -c bioconda snakemake

After this, you'll probably go through the same problems that I have, and for which I still have no working solution. I know barely anything about snakemake and mono.

pillepalle123 commented 5 years ago

thanks a lot! That helped! I feel I am getting closer. Now I can continue with the next issue :)

Building DAG of jobs... Pulling singularity image docker://mono:5.16.0.226. WorkflowError: Failed to pull singularity image from docker://mono:5.16.0.226: WARNING: Authentication token file not found : Only pulls of public images will succeed INFO: Starting build... FATAL: Unable to pull docker://mono:5.16.0.226: conveyor failed to get: Error reading manifest 5.16.0.226 in docker.io/library/mono: manifest unknown: manifest unknown

You didn't happen to get this as well?

Best

edit: solved it, thank you

stjacqrm commented 5 years ago

Hello iquasere,

I'm having the same issue you were having last year. I've tried running snakemake with the --use-singularity --singularity-args flags, but snakemake is unable to recognize them. Were you ever able to find a solution?

edit- I'm not using snakemake for the same tool that you are, if that matters at all

iquasere commented 5 years ago

Hello stjacqrm,

I never walked outside of first steps with UltraQuant, and never found a workaround for this issue. Pillepalle, however, seems to have beaten the issues concerning UltraQuant, and as such, might have solved my problem as well.

stjacqrm commented 5 years ago

Thanks!

I'll take a look at their solutions to see if I can make it work!