search

Wytamma / snk

Snakemake workflow management system and CLI generation tool
https://snk.wytamma.com
MIT License
31 stars 2 forks source link

Make conda usage optional #45

Closed pbousquets closed 12 months ago

pbousquets commented 1 year ago

Hi @Wytamma,

I've decided to create clean issue related to my previous comment (https://github.com/Wytamma/snk/issues/20#issuecomment-1781025618_).

I developed the code in a server where I had conda and mamba, so I never encountered any problem. However, when I moved to another computer with no conda installation, it didn't work. The error is this one, and is coming from snakemake: 

/usr/bin/bash: conda: command not found
Full Traceback (most recent call last):
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/__init__.py", line 757, in snakemake
    success = workflow.execute(
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/workflow.py", line 1091, in execute
    raise e
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/workflow.py", line 1087, in execute
    success = self.scheduler.schedule()
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/scheduler.py", line 602, in schedule
    self.run(runjobs)
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/scheduler.py", line 651, in run
    executor.run_jobs(
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 155, in run_jobs
    self.run(
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 535, in run
    future = self.run_single_job(job)
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 588, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 576, in job_args_and_prepare
    self.workflow.conda_base_path,
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/workflow.py", line 299, in conda_base_path
    return Conda().prefix_path
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/deployment/conda.py", line 667, in __init__
    shell.check_output(self._get_cmd(f"conda info --json"), text=True)
  File "/home/pablo/.local/share/hatch/env/virtual/mypackage/PWKu738T/mypackage/lib/python3.8/site-packages/snakemake/shell.py", line 61, in check_output
    return sp.check_output(cmd, shell=True, executable=executable, **kwargs)
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'conda info --json' returned non-zero exit status 127.

The reason why I think it is related to snk is because of this piece of code: https://github.com/Wytamma/snk/blob/45ef4297e4efc1b327a251e27dd5ea66341116e0/snk/cli/subcommands/run.py#L173-L179

I ran the pipeline in verbose mode and, indeed, it is asking snakemake to activate conda, even if I don't add any conda environment in my Snakefile:

snakemake --verbose --use-conda --conda-prefix=/mnt/d/Lab/mypackage/mypackage/src/mypackage/.conda --cores=all --snakefile=/mnt/d/Lab/mypackage/mypackage/src/mypackage/workflow/Snakefile --conda-frontend=conda --nolock --config input_dir=/mnt/d/Lab/mypackage/mypackage/example output=test mode=trees patientInfo=/mnt/d/Lab/mypackage/example/meta.csv  workdir=test threads=1

Could you have a look and check if the same happens on any of your packages? I think that the fix could even be easily automated, by checking if conda or mamba binaries are available in $PATH. If so, you can activate the piece of code I referenced above, and deactivate it otherwise. This way the user wouldn't even need to explicitly activate it.

Thanks a lot!

Pablo

Wytamma commented 12 months ago

Hey @pbousquets!

Thanks for the issue! I've solved it in https://github.com/Wytamma/snk/releases/tag/v0.16.0. the conda flags will only be added if the conda command is available. You can require conda (pipeline won't run without conda) by adding require_conda: true to the snk.yaml config.

-W