Problems with Singularity definition file

[fbartusch]

Hello,

A user asked if we can install HiC-Pro on our HPC-cluster. So I tried to build a Singularity container from the definition file and experienced some problems and odd things:

In the %pre section you try to install debootstrap with apt-get. As I'm working on CentOS, this throws an error when building the image. The build also worked without that section.

You install Miniconda 3 and then try later to install Python 2.7.15. This does not work on my machine and I'm quite sure that it also does not work for other people. I solved this by installing the latest Miniconda2 and and removed the additional python installation.

Also, conda was not able to resolve some dependencies. I think the problem is, that you place each package on a new line and also have the restriction for a specific version. For me it worked to place packages from the same repository on the same line. Maybe it would be better to provide a file describing the conda environment in a file and install the dependencies from this file.

Here's the modified Singularity definition file that worked for me. Maybe this feedback is helpful for you and others.

Best wishes, Felix

BootStrap: docker
From: ubuntu:latest

%labels
    AUTHOR Nicolas Servant

%pre
    #apt-get install -y debootstrap

%post
    apt-get update
    apt-get install -y wget
    apt-get install -y gzip
    apt-get install -y bzip2
    apt-get install -y curl
    apt-get install -y unzip
    apt-get install -y vim

    ## g++
    apt-get install -y build-essential

    # install anaconda
    if [ ! -d /usr/local/anaconda ]; then
        wget https://repo.anaconda.com/miniconda/Miniconda2-latest-Linux-x86_64.sh \
             -O ~/anaconda.sh && \
        bash ~/anaconda.sh -b -p /usr/local/anaconda && \
        rm ~/anaconda.sh
    fi

    # set anaconda path
    export PATH=$PATH:/usr/local/anaconda/bin

    conda update conda

    conda config --add channels r
    conda config --add channels defaults
    conda config --add channels conda-forge
    conda config --add channels bioconda

    # Let us save some space
    conda clean --packages -y

    # external tools
    echo "Installing dependancies ... "
    conda install -y bowtie2
    conda install -y samtools

    # Python (>2.7) with *pysam (>=0.8.3)*, *bx(>=0.5.0)*, *numpy(>=1.8.2)*, and *scipy(>=0.15.1)* libraries
    #conda install -y c conda-forge python=2.7.15
    conda install -y -c anaconda scipy=1.2.1 numpy=1.16.3
    conda install -y -c bcbio bx-python=0.8.2
    conda install -y -c bioconda pysam=0.15.2

    # Install R
    conda update readline
    ##conda install -c conda-forge readline=6.2
    conda install -c r r-base=3.5.1 r-ggplot2=2.2.1 r-rcolorbrewer=1.1_2 r-gridbase=0.4_7

    # Install MultiQC
    conda install -c bioconda multiqc=1.7

    # Install HiC-pro
    VERSION=2.11.4
    echo "Installing HiC-Pro release ${VERSION} ..."
    wget https://github.com/nservant/HiC-Pro/archive/v${VERSION}.tar.gz
    tar xzf v${VERSION}.tar.gz
    cd HiC-Pro-${VERSION}
    make configure
    make install

    # Let us save some space
    conda clean --packages -y
    conda clean --all -y
    rm -rf /usr/local/anaconda/pkgs
    apt-get clean

%test
    INSTALLED_HICPRO_VERSION=$(find /usr/local/bin -name HiC-Pro | xargs dirname)
    $INSTALLED_HICPRO_VERSION/HiC-Pro -h

%environment
    export PATH=$PATH:/usr/local/anaconda/bin
    INSTALLED_VERSION=$(find /usr/local/bin -name HiC-Pro | xargs dirname)
    export PATH=$PATH:$INSTALLED_VERSION
    export LANG=C

[nservant]

Hi Felix, Many thanks for your feedback. Which HiC-Pro version did you use ? I updated everything on the devel branch a couple of months ago, upgrading most of the packages, and including a conda yml file, etc. I will try to release the devel branch before the end of the year. Best Nicolas

[fbartusch]

Hi Nicolas,

Thans for your reply. I used the file in the definition file from the master branch and HiC-Pro 2.11.4. Now I saw that most problems I encountered seems to be fixed in the Singularity definition file in the devel branch. Maybe you can get rid of the pre section, as this hinders the build of the container on non-Debian based systems. I adverted the new version to our user and we will try it out when it's released. Best Felix