Dockerfile to build gearshifft

[ChristianKniep]

I am looking for an example to run a benchmark that leverages multiple hardware platforms. CPU instruction and GPU support. To make this work I would like to create Dockerfiles to build different version. Any chance you have a Dockerfile already?

[ChristianKniep]

Starting with this now...

ARG DOCKER_REGISTRY=docker.io
ARG FROM_IMG_REPO=qnib
ARG FROM_IMG_NAME="uplain-init"
ARG FROM_IMG_TAG="bionic-20180821_2018-09-27"
ARG FROM_IMG_HASH=""
FROM ${DOCKER_REGISTRY}/${FROM_IMG_REPO}/${FROM_IMG_NAME}:${FROM_IMG_TAG}${DOCKER_IMG_HASH}

ENV DEBIAN_FRONTEND=noninteractive \
    DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt update \
 && apt install -y --no-install-recommends \
                build-essential \
                ca-certificates \
                cmake \
                git

WORKDIR /opt
RUN git clone https://github.com/mpicbg-scicomp/gearshifft.git
WORKDIR /opt/gearshifft/release
RUN cmake -DGEARSHIFFT_USE_SUPERBUILD=ON \
      -DGEARSHIFFT_SUPERBUILD_EXT_INBUILD=OFF \
      -DGEARSHIFFT_SUPERBUILD_EXT_DIR=$HOME/sources/deps \
      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_Boost=ON \
      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_CLFFT=ON \
      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_FFTW=ON \
      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_ROCFFT=OFF \
      -DGEARSHIFFT_USE_STATIC_LIBS=ON \
      ..
RUN make -j 4

[psteinb]

I hope your base image contains a compiler, otherwise I don’t see a problem here.

[emmenlau]

We build in Docker for a variety of Linux, no problems so far. However we do not execute inside Docker. It may be interesting to see how stable the Docker GPU bindings are...

[psteinb]

my initial tests with deep learning and singulairty suggest a minimal impact (bare metal vs. in-container load). but I must say, that the performance profile of FFTs and DL tend to diverge.

[ChristianKniep]

@emmenlau Do you have a Dockerfile handy? It seems that my ubuntu 18.04 image is not suited. Using GPUs within containers is not to complex on the surface. Mapping /dev/nvidia{0,ctl,-uwm is the easy part - apart from that one needs to make sure that the kernel driver on the host and the user-land driver within the container-FS are matching. I elaborate on that here: http://www.qnib.org/2019/02/14/manifest-list-to-pick-optimized-images/

@psteinb As you can optimize the user-land of the container compared to the one on the host, I'd argue you can even exceed the performance of bare metal. Most of the time the bare-metal installation is a compromise of different use-cases and as such not optimised for a specific one. Container can help there. I had a benchmark a long time ago showing that (http://www.qnib.org/qnibterminal/2014/12/02/Containerized-MPI-workloads-Interview/), needs some reproducing tho - that is where you guys come in. :)

Happy to jump on a quick zoom call to discuss how I would like to promote a benchmark to optimise for host configurations.

[ChristianKniep]

Oh and in that context; would love to see you participating at the ISC... https://easychair.org/cfp/hpcw2019

[ChristianKniep]

Unfortunately, I am still not able to build gearshifft... Current approach with CentOS7:

ARG DOCKER_REGISTRY=docker.io
ARG FROM_IMG_REPO=library
ARG FROM_IMG_NAME="centos"
ARG FROM_IMG_TAG="7.6.1810"
ARG FROM_IMG_HASH=""
FROM ${DOCKER_REGISTRY}/${FROM_IMG_REPO}/${FROM_IMG_NAME}:${FROM_IMG_TAG}${DOCKER_IMG_HASH}

RUN yum update -y \
 && yum install -y git
WORKDIR /opt
RUN git clone https://github.com/mpicbg-scicomp/gearshifft.git
WORKDIR /opt/gearshifft/release
RUN yum -y install epel-release \
 && yum install -y cmake3
RUN yum install -y make
RUN yum groupinstall -y "Development Tools"
RUN yum install -y which
RUN yum install -y boost169-devel
#RUN cmake3 -DGEARSHIFFT_USE_SUPERBUILD=OFF \
#      -DGEARSHIFFT_SUPERBUILD_EXT_INBUILD=OFF \
#      -DGEARSHIFFT_SUPERBUILD_EXT_DIR=$HOME/sources/deps \
#      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_Boost=OFF \
#      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_CLFFT=OFF \
#      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_FFTW=OFF \
#      -DGEARSHIFFT_SUPERBUILD_EXT_DOWNLOAD_ROCFFT=OFF \
#      -DGEARSHIFFT_USE_STATIC_LIBS=OFF \
RUN cmake3      ..
RUN make -j8

Provides me this outcome:

-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake3/Modules/FindBoost.cmake:2100 (message):
  Unable to find the requested Boost libraries.

  Unable to find the Boost header files.  Please set BOOST_ROOT to the root
  directory containing Boost or BOOST_INCLUDEDIR to the directory containing
  Boost's headers.
Call Stack (most recent call first):
  CMakeLists.txt:74 (find_package)

-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
FFTWWrappers will prefer libraries with no prefix and no suffix
FFTWWrappers was not able to find fftw3.h in
FFTWWrappers was not able to find mkl_intel_lp64 in
FFTWWrappers was not able to find mkl_intel_thread in
FFTWWrappers was not able to find mkl_core in
CMake Warning at CMakeLists.txt:430 (message):
  No FFT backend found.

-- Configuring incomplete, errors occurred!
See also "/opt/gearshifft/release/CMakeFiles/CMakeOutput.log".
See also "/opt/gearshifft/release/CMakeFiles/CMakeError.log".
The command '/bin/sh -c cmake3      ..' returned a non-zero code: 1
>> Build failed...

[emmenlau]

Dear @ChristianKniep I don't have anything that could easily help you, because we use several vanilla Linux Docker images and build all our third party dependencies by hand. So I can only confirm the build of gearshifft works on a lot of Docker images, but I don't have ready-made images or instructions. But I may still be able to provide a little bit of help: When we build gearshifft, boost is in a non-standard directory, and we set the cmake option -DBOOST_ROOT="/path/to/boost/installation". It seems your CentOS may need the same?

[boegel]

@ChristianKniep Your missing both Boost and FFTW in the container from the looks of it.

Or you can use EasyBuild instead to install gearshifft and all the required deps, see https://github.com/easybuilders/easybuild-easyconfigs/pull/8482.

If you have EasyBuild installed, eb --from-pr 8482 should do the trick...

[ChristianKniep]

I refined my Dockerfile and create one that is able to build CPU images. Still WiP to make a broader benchmark possible.

https://github.com/qnib/uplain-gearshifft/blob/master/build.sh