Build from source inside Dockerfile

[benoitberanger]

Describe the bug I try to install ANTsPy in a Docker, but cannot build from source.

In the Dockerfile :

FROM python:3.12.0-slim

RUN  apt-get update && apt-get install -y git cmake g++ libhdf5-dev libxml2-dev libxslt1-dev libboost-all-dev libfftw3-dev libpugixml-dev
RUN  mkdir -p /opt/code

# more unrelated lines...

# ANTsPyX 
RUN cd /opt/code && \ 
         git clone https://github.com/antsx/antspy && \ 
         cd antspy && \ 
         python -m pip --no-cache-dir install . 

And during docker build :

Building wheels for collected packages: antspyx
  Building wheel for antspyx (pyproject.toml): started
  Building wheel for antspyx (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  × Building wheel for antspyx (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      *** scikit-build-core 0.10.5 using CMake 3.30.2 (wheel)
      *** Configuring CMake...
      loading initial cache file build/cp312-cp312-linux_x86_64/CMakeInit.txt
      CMake Error at /tmp/pip-build-env-n_4zfa5_/normal/lib/python3.12/site-packages/cmake/data/share/cmake-3.30/Modules/CMakeDetermineCXXCompiler.cmake:48 (message):
        Could not find compiler set in environment variable CXX:

        g++.

      Call Stack (most recent call first):
        CMakeLists.txt:3 (project)

      CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
      -- Configuring incomplete, errors occurred!

      *** CMake configuration failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for antspyx
Failed to build antspyx
ERROR: Could not build wheels for antspyx, which is required to install pyproject.toml-based projects

[notice] A new release of pip is available: 23.2.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
The command '/bin/sh -c cd /opt/code &&     git clone https://github.com/antsx/antspy &&     cd antspy &&     python -m pip --no-cache-dir install .' returned a non-zero code: 1

ANTsPy installation:

• System details [ Docker ]

• ANTsPy version: [ 504a43a5de1a8c1b509a048af0a1014eb33c4f51 : latest commit, no available pre-compiled binaries using pip install]

• Installation type: [ build from source in a Dockerfile ]

Thank you for your help, Benoît

[benoitberanger]

Some more lines in the middle of the Dockerfile show that it's not a Python our PIP problem :

# Python MRD library
RUN pip3 install h5py==3.10.0 ismrmrd==1.14.1

RUN cd /opt/code && \
    git clone https://github.com/ismrmrd/ismrmrd-python-tools.git && \
    cd /opt/code/ismrmrd-python-tools && \
    pip3 install --no-cache-dir .

# matplotlib is used by rgb.py and provides various visualization tools including colormaps
# pydicom is used by dicom2mrd.py to parse DICOM data
RUN pip3 install --no-cache-dir matplotlib==3.8.2 pydicom==2.4.3 pynetdicom==2.0.2

Just to be sure tried in Dockerfile pip3 --no-cache-dir install . instead of python -m pip --no-cache-dir install .. Same result, still stopping at the CMake error.

[cookpa]

You would also need libpng-dev in your apt call.

Strange that it doesn't find g++ though. Perhaps some of those unrelated lines are altering the PATH?

[benoitberanger]

Thank you for your help.

You are correct, something is wrong in my build process somewhere. Because with a simpler Dockerfile like this one bellow, it compiles correctly :

FROM python:3.12.0-slim

RUN  apt-get update && apt-get install -y git cmake g++ libhdf5-dev libxml2-dev libxslt1-dev libboost-all-dev libfftw3-dev libpugixml-dev
RUN  mkdir -p /opt/code

# Python MRD library
RUN pip3 install h5py==3.10.0 ismrmrd==1.14.1

RUN cd /opt/code && \
    git clone https://github.com/ismrmrd/ismrmrd-python-tools.git && \
    cd /opt/code/ismrmrd-python-tools && \
    pip3 install --no-cache-dir .

# matplotlib is used by rgb.py and provides various visualization tools including colormaps
# pydicom is used by dicom2mrd.py to parse DICOM data
RUN pip3 install --no-cache-dir matplotlib==3.8.2 pydicom==2.4.3 pynetdicom==2.0.2

# ANTsPyX 
RUN  apt-get update && apt-get install -y libpng-dev
RUN cd /opt/code && \ 
    git clone https://github.com/antsx/antspy && \ 
    cd antspy && \ 
    python -m pip --no-cache-dir install . 

[benoitberanger]

My guess is the problem comes from the multi-stage docker build process, it's not a ANTsPy problem. A bit more detail :

First step, build this docker https://github.com/kspaceKelvin/python-ismrmrd-server/blob/master/docker/Dockerfile, which is used as starting point right after.

Second step, build a docker for the specific app (in this case, using ANTsPy) :

import python-ismrmrd-server as starting point
FROM python-ismrmrd-server

# mandatory for OpenRecon (see OR documentation)
LABEL "com.siemens-healthineers.magneticresonance.openrecon.metadata:1.1.0"="{encoded_json_content}"

# copy the .py module
COPY {path_app_file}  /opt/code/python-ismrmrd-server

# ANTsPyX
RUN apt-get update && apt-get install -y g++ cmake libpng-dev
#    need to add g++ and cmake... dont know why it's forgotten
RUN cd /opt/code && \
        git clone https://github.com/antsx/antspy && \
        cd antspy && \
        pip3 --no-cache-dir install .

# new CMD line
{cmdline}

[cookpa]

g++ is not forgotten, it is deliberately left out to reduce the final container size and enhance security. It's pretty common to do this with multi-stage builds. In the building stage, compilers and other development tools are used to build some software. Then the run time layer is initialized, often from the same base image, and the binaries are copied over.

This lets you build things and then copy them to a runtime without having all the build tools, package caches, etc in the final layer.

FROM python:3.12.0-slim AS build
# install cmake, g++, etc
# build some tool
FROM python:3.12.0-slim AS runtime
# We now have a copy of the base image, without anything we did above
# use COPY to copy over compiled binaries from the build image

[benoitberanger]

Ok so it is a Docker building process. Thank you for the clarification.