search

gbouras13 / hybracter

Automated long-read first bacterial genome assembly tool implemented in Snakemake using Snaketool.
MIT License
92 stars 7 forks source link

Mamba not detected with snakemake #85

Open dinmoz opened 4 weeks ago

dinmoz commented 4 weeks ago

Describe the bug Mamba not detected in snakemake when hybracter is run on the cluster via slurm. hybracterENV installed with miniforge3.

To Reproduce

  1. conda activate hybracterENV in the submission script, then hybracter long for pacbio reads (including --no_medaka --flyeModel --pacbio-hifi ).
  2. Error message: CreateCondaEnvironmentException: The 'mamba' command is not available in the shell /bin/bash that will be used by Snakemake. You have to ensure that it is in your PATH, e.g., first activating the conda base environment with conda activate base.The mamba package manager (https://github.com/mamba-org/mamba) is a fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed with conda install -n base -c conda-forge mamba. If you still prefer to use conda, you can enforce that by setting --conda-frontend conda. ERROR: Snakemake failed

Additional context Interestingly, mamba is detected prior to activating the hybracter environment (which mamba), but snakemake fails saying it cannot find mamba. Also tried --conda-frontend conda instead, but receive the following error: Activating conda environment: ../../../miniforge3/envs/hybracterENV2/lib/python3.12/site-packages/hybracter/workflow/conda/07b2ed8e593a2afce4f34d72461f46bd_ /bin/bash: /usr/bin/activate: No such file or directory /bin/bash: line 1: filtlong: command not found

dinmoz commented 4 weeks ago

Should also mention that previously hybracter test long ran successfully, as well as hybracter long-single on the head node. So hybracter appears to work outside of slurm/compute node.

gbouras13 commented 4 weeks ago

Hi @dinmoz ,

This is a really strange error that I am not sure is related to hybracter per se (seems like a cluster problem).

Does your cluster have singularity support? I would highly recommend running it with Singularity if you can for your case, it will probably be easier than trying to debug this error!

https://github.com/gbouras13/hybracter?tab=readme-ov-file#container

George

dinmoz commented 4 weeks ago

Thanks @gbouras13 for your prompt reply! Yes, we have singularity support, I figured it may be easier to run the container as well. Currently pulling the image which is admittedly taking some time. Hoping to test this out soon!

gbouras13 commented 4 weeks ago

It is a very large container, apologies for that - I guess it the only way to make sure all the conda env are installed.

Let me know how you go!

George

nbat64 commented 2 weeks ago

Hi same issue HPC slurm and hybracter installed in mamba env. It is working when running on the frontend but failed when submitted. So a singularity image is the solution?

dinmoz commented 2 weeks ago

@nbat64 The container image worked for me, ran fine via slurm/cluster.

nbat64 commented 2 weeks ago

Ok thanks. No I have error 

 ERROR: missing database file /dlocal/home/2019013/Databases/hybracter/plsdb_2023_11_03_v2.msh
 ERROR: missing database file /dlocal/home/2019013/Databases/hybracter/plsdb_2023_11_03_v2.tsv

    FATAL: One or more database files is missing.
    Please run 'hybracter install' to download and install the missing database files.

SystemExit in file /opt/miniforge3/lib/python3.10/site-packages/hybracter/workflow/rules/preflight/functions.smk, line 101:
1
  File "/opt/miniforge3/lib/python3.10/site-packages/hybracter/workflow/hybrid.smk", line 46, in <module>
  File "/opt/miniforge3/lib/python3.10/site-packages/hybracter/workflow/rules/preflight/functions.smk", line 101, in check_db
[2024:08:24 13:09:59] ERROR: Snakemake failed

I don't understand as the files are present (but where installed before using the mamba version)?

gbouras13 commented 2 weeks ago

Hi @nbat64,

To be clear, is this error with the container? You shouldn't need to specify a database directory (I guess you did -d /dlocal/home/2019013/Databases/hybracter ?) with the container as it has an default database installed inside. https://github.com/gbouras13/hybracter?tab=readme-ov-file#container

George

gbouras13 commented 2 weeks ago

And for any future people reading this, if your cluster supports singularity/apptainer, I'd recommend the container over a conda install, because it "usually works" (mostly) once you download/make the .sif file, and clusters usually prefer these solutions to conda (which makes lots of files).

George

nbat64 commented 2 weeks ago

Hello, Thanks for the reply. Yes, I am now using singularity image. I remove the two arguments --databases and --contaminants and now it's working well. Best, Nicolas