bismark_genome_preparation failture: `/usr/bin/env: 'python': No such file or directory`

[PanZiwei]

Hi, I am trying to use bismark_genome_preparation in bismark via singularity to prepare the hg38 genome on the cluster, but I failed to launch the bowtier2 index at step3(See information below). And I checked the output, it only produced two files: genome_mfa.CT_conversion.fa and genome_mfa.GA_conversion.fa, no BS_*.fa files are produced.

I checked your answers https://github.com/FelixKrueger/Bismark/issues/449#issuecomment-898968970 to another similar question, but I am sure that the memory should be enough since I asked 16 CPU with 20G.

Do you have an idea on this issue? Thanks!

(base) [c-panz@sumner065 sif]$ srun --pty -q batch -t 12:0:0  --cpus-per-task=16 --mem-per-cpu=20G bash -i
(base) [c-panz@sumner065 sif]$ singularity exec bismark_latest.sif bismark_genome_preparation --bowtie2 /fastscratch/c-panz/nextflow/reference/
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Writing bisulfite genomes out into a single MFA (multi FastA) file

Bisulfite Genome Indexer version v0.22.3 (last modified: 14 April 2019)

Step I - Prepare genome folders - completed

Total number of conversions performed:
C->T:   598683433
G->A:   600854940

Step II - Genome bisulfite conversions - completed

Bismark Genome Preparation - Step III: Launching the Bowtie 2 indexer
Please be aware that this process can - depending on genome size - take several hours!
/usr/bin/env: 'python': No such file or directory
Parent process: Failed to build index
(base) [c-panz@sumner065 sif]$ /usr/bin/env: 'python': No such file or directory

[FelixKrueger]

Hi @PanZiwei

the latest version of thebowtie2-build process requires that you've got Python (3?) in your PATH as well. If you could add that that to your singularity image, it should work (I hope)...

[PanZiwei]

Hi @FelixKrueger

Thank you so much for your quick response.

I checked the python version and it is 3.7.10 on the cluster. So you are suggesting that I have to add the python path into the singularity image?

Do you have an official docker image singularity? I download the image from https://hub.docker.com/r/kerstenbreuer/bismark, maybe their image didn't contain python3...

(base) [c-panz@sumner065 sif]$ python3 --version
Python 3.7.10
(base) [c-panz@sumner065 sif]$ python --version
Python 3.7.10

[FelixKrueger]

I'm afraid I don't have a Docker or Singularity image myself. As far as I know, Bowtie2 alignments do not need a working version of Python, but the indexing does (not sure it is mentioned as a dependency, it is probably some wrapper script...).

I guess the options would be to modify the Docker image to include Python, or ask Kersten Breuer to update theirs. Or, if you've got Python and Bowtie2 installed on your cluster, as a one-off you could clone or download Bismark (https://github.com/FelixKrueger/Bismark/releases) and then run the same command (bismark_genome_preparation --verbose /fastscratch/c-panz/nextflow/references/) offline.

I guess after this the Singularity image you were using will work just fine.

[PanZiwei]

I checked the downloaded image and found that it indeed lacks python3 package.

Maybe a naive question, how do can I set up Bowtie2 in the PATH environment after I downloaded Bismark successfully? Do you mean that the Bowtie2 has to be under the Bismark folder for usage? Thanks!

[FelixKrueger]

You can install Bowtie2 (http://bowtie-bio.sourceforge.net/bowtie2/index.shtml) via conda, or download and extract the zip file from: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/2.4.4

Once you have done that, add that folder to the PATH, or you can point bismark_genome_preparation to the Bowtie2 installation folder using --path_to_aligner (or so, please see --help for details). It doesn't need to be in the same folder as Bismark itself.

[PanZiwei]

Hi,

Thanks for the suggestion. I finally finished the preparation step with your suggestion.

To be more specific, I create a conda environment with bowtie2 installed and relocate the conda environment dir to run it.

[FelixKrueger]

Excellent, glad you got it to work!