ANTsX / ANTs

Advanced Normalization Tools (ANTs)
Apache License 2.0
1.21k stars 381 forks source link

Request: remove git dependency #1610

Closed pvelasco closed 12 months ago

pvelasco commented 1 year ago

When did the error occur? [X] CMake configuration (cmake / ccmake) [ ] Compilation (make) [ ] Installation (make install)

Build environment

ANTs version 2.4.2

Build configuration and logs

Additional context

Is it possible to remove the git dependency? I'm building a Docker image with ANTs and I'm trying to install as few dependencies as possible. It seems to me that there should be other ways of extracting the version info than using git...

Thanks.

cookpa commented 1 year ago

If you have a tagged release, the version information should be read directly from Version.cmake. You won't be able to do a superbuild, because that needs git to checkout ITK.

But the version information itself should not require git. It's possible that there's a bug, or maybe v2.4.2 didn't have all the necessary changes.

If you can upload the full details and build output, I'll take a look.

You might have already considered this, but could you build ANTs in a build layer and then copy the executables to a final install layer without git installed? This is how the ANTs Dockerfile works: the build layer has gcc, git, and all the development stuff, then the executables are copied to a smaller runtime layer without all that stuff.

pvelasco commented 1 year ago

Thanks for your response, @cookpa ,

Here is my `Dockerfile` ```dockerfile FROM python@sha256:ffc6cb648d6993e7c90abb95c2481eb688a6842dfac29bf19e3755454632daa2 as base RUN apt-get update \ && apt-get install -y --no-install-recommends \ apt-transport-https \ bc \ build-essential \ ca-certificates \ cmake \ gnupg \ ninja-build \ software-properties-common \ unzip \ wget \ && rm -rf /var/lib/apt/lists/* \ && mkdir -p /tmp/ants/build # Install ANTs ENV ANTS_VERSION "v2.4.2" RUN cd /tmp/ants/ \ && wget -q https://github.com/ANTsX/ANTs/archive/refs/tags/${ANTS_VERSION}.zip -O ants.zip \ && unzip ants.zip \ && rm ants.zip \ && cd /tmp/ants/build \ && cmake \ -DCMAKE_INSTALL_PREFIX=/opt/ants \ -DBUILD_SHARED_LIBS=OFF \ -DUSE_VTK=OFF \ -DSuperBuild_ANTS_USE_GIT_PROTOCOL=OFF \ -DBUILD_TESTING=OFF \ -DRUN_LONG_TESTS=OFF \ -DRUN_SHORT_TESTS=OFF \ /tmp/ants/ANTs-${ANTS_VERSION#v} 2>&1 | tee cmake.log \ && make -j 4 2>&1 | tee build.log \ && cd ./ANTS-build \ && make install 2>&1 | tee install.log \ # Move missing scripts to binary \ && mv /tmp/ants/ANTs-${ANTS_VERSION#v}/Scripts/* /opt/ants/bin/ \ && rm -fr /tmp/ants ```

Then, I run docker build -t deleteme/test .

And this is the output ```shell [+] Building 32.0s (6/6) FINISHED => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 1.29kB 0.0s => [internal] load metadata for docker.io/library/python@sha256:ffc6cb648d6993e7c90abb95c2481eb688a6842dfac29bf19e3755454632daa2 0.0s => CACHED [1/3] FROM docker.io/library/python@sha256:ffc6cb648d6993e7c90abb95c2481eb688a6842dfac29bf19e3755454632daa2 0.0s => [2/3] RUN apt-get update && apt-get install -y --no-install-recommends apt-transport-https bc build-essential 26.1s => ERROR [3/3] RUN cd /tmp/ants/ && wget -q https://github.com/ANTsX/ANTs/archive/refs/tags/${ANTS_VERSION}.zip -O ants.zip && unzip ant 5.6s ------ > [3/3] RUN cd /tmp/ants/ && wget -q https://github.com/ANTsX/ANTs/archive/refs/tags/${ANTS_VERSION}.zip -O ants.zip && unzip ants.zip && rm ants.zip && cd /tmp/ants/build && cmake -DCMAKE_INSTALL_PREFIX=/opt/ants -DBUILD_SHARED_LIBS=OFF -DUSE_VTK=OFF -DSuperBuild_ANTS_USE_GIT_PROTOCOL=OFF -DBUILD_TESTING=OFF -DRUN_LONG_TESTS=OFF -DRUN_SHORT_TESTS=OFF /tmp/ants/ANTs-${ANTS_VERSION#v} 2>&1 | tee cmake.log && make -j 4 2>&1 | tee build.log && cd ./ANTS-build && make install 2>&1 | tee install.log && mv /tmp/ants/ANTs-${ANTS_VERSION#v}/Scripts/* /opt/ants/bin/ && rm -fr /tmp/ants: #4 1.371 Archive: ants.zip #4 1.371 6c729710ab2f545d0ab8f07393d180eb3eeb1900 #4 1.371 creating: ANTs-2.4.2/ #4 1.372 inflating: ANTs-2.4.2/.clang-format #4 1.372 creating: ANTs-2.4.2/.github/ #4 1.372 extracting: ANTs-2.4.2/.github/FUNDING.yml #4 1.372 creating: ANTs-2.4.2/.github/ISSUE_TEMPLATE/ #4 1.372 inflating: ANTs-2.4.2/.github/ISSUE_TEMPLATE/bug_report.md #4 1.372 inflating: ANTs-2.4.2/.github/ISSUE_TEMPLATE/build_errors.md #4 1.372 inflating: ANTs-2.4.2/.github/ISSUE_TEMPLATE/config.yml #4 1.372 inflating: ANTs-2.4.2/.github/ISSUE_TEMPLATE/feature_request.md #4 1.372 inflating: ANTs-2.4.2/.github/ISSUE_TEMPLATE/other_questions.md #4 1.372 inflating: ANTs-2.4.2/.github/pull_request_template.md #4 1.372 inflating: ANTs-2.4.2/.github/stale.yml #4 1.373 creating: ANTs-2.4.2/.github/workflows/ #4 1.373 inflating: ANTs-2.4.2/.github/workflows/ci-docker.yml #4 1.373 inflating: ANTs-2.4.2/.github/workflows/release-binaries.yml #4 1.373 inflating: ANTs-2.4.2/.github/workflows/release-docker-binaries.yml #4 1.373 inflating: ANTs-2.4.2/.gitignore #4 1.373 extracting: ANTs-2.4.2/.htaccess #4 1.373 inflating: ANTs-2.4.2/.mailmap #4 1.373 inflating: ANTs-2.4.2/.travis.yml #4 1.373 inflating: ANTs-2.4.2/ANTS.cmake #4 1.373 inflating: ANTs-2.4.2/ANTSCopyright.txt #4 1.374 inflating: ANTs-2.4.2/ANTSnightly.sh #4 1.374 inflating: ANTs-2.4.2/ANTsVersionConfig.h.in #4 1.374 creating: ANTs-2.4.2/CMake/ #4 1.374 inflating: ANTs-2.4.2/CMake/ANTSExternalData.cmake #4 1.374 inflating: ANTs-2.4.2/CMake/CCache.cmake #4 1.374 inflating: ANTs-2.4.2/CMake/ExternalData_config.cmake.in #4 1.374 inflating: ANTs-2.4.2/CMake/GetGitRevisionDescription.cmake #4 1.374 inflating: ANTs-2.4.2/CMake/ITKSetStandardCompilerFlags.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/ITK_CheckCCompilerFlag.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/ITK_CheckCXXCompilerFlag.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/PreventInBuildInstalls.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/PreventInSourceBuilds.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/ProjectSourceVersion.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/SlicerMacroCheckExternalProjectDependency.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/SlicerMacroEmptyExternalProject.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/itkCheckSourceTree.cmake #4 1.375 inflating: ANTs-2.4.2/CMake/pre-commit #4 1.376 inflating: ANTs-2.4.2/CMakeLists.txt #4 1.376 inflating: ANTs-2.4.2/COPYING.txt #4 1.376 inflating: ANTs-2.4.2/CTestConfig.cmake #4 1.376 inflating: ANTs-2.4.2/CTestCustom.cmake #4 1.376 inflating: ANTs-2.4.2/Common.cmake #4 1.376 inflating: ANTs-2.4.2/Dockerfile #4 1.376 creating: ANTs-2.4.2/Examples/ #4 1.376 inflating: ANTs-2.4.2/Examples/ANTS.cxx [...] snip [...] #4 1.481 inflating: ANTs-2.4.2/forhtml/README #4 1.481 inflating: ANTs-2.4.2/forhtml/ants.bib #4 1.482 inflating: ANTs-2.4.2/forhtml/ants_progress_report.Rmd #4 1.483 extracting: ANTs-2.4.2/forhtml/nazca-1.jpg #4 1.490 inflating: ANTs-2.4.2/forhtml/push-website #4 1.490 inflating: ANTs-2.4.2/index.html #4 1.490 inflating: ANTs-2.4.2/index.md #4 1.567 -- The C compiler identification is GNU 10.2.1 #4 1.630 -- The CXX compiler identification is GNU 10.2.1 #4 1.639 -- Detecting C compiler ABI info #4 1.728 -- Detecting C compiler ABI info - done #4 1.740 -- Check for working C compiler: /usr/bin/cc - skipped #4 1.740 -- Detecting C compile features #4 1.740 -- Detecting C compile features - done #4 1.744 -- Detecting CXX compiler ABI info #4 1.838 -- Detecting CXX compiler ABI info - done #4 1.850 -- Check for working CXX compiler: /usr/bin/c++ - skipped #4 1.850 -- Detecting CXX compile features #4 1.851 -- Detecting CXX compile features - done #4 1.853 -- Not using CCache #4 1.853 -- Setting build type to 'Release' as none was specified. #4 2.065 -- Performing Test C_HAS_WARNING-Wno-uninitialized #4 2.141 -- Performing Test C_HAS_WARNING-Wno-uninitialized - Success #4 2.142 -- Performing Test C_HAS_WARNING-Wno-unused-parameter #4 2.226 -- Performing Test C_HAS_WARNING-Wno-unused-parameter - Success #4 2.227 -- Performing Test C_HAS_WARNING-Wall #4 2.312 -- Performing Test C_HAS_WARNING-Wall - Success #4 2.313 -- Performing Test C_HAS_WARNING-Wno-long-double #4 2.392 -- Performing Test C_HAS_WARNING-Wno-long-double - Failed #4 2.393 -- Performing Test C_HAS_WARNING-Wcast-align #4 2.468 -- Performing Test C_HAS_WARNING-Wcast-align - Success #4 2.469 -- Performing Test C_HAS_WARNING-Wdisabled-optimization #4 2.544 -- Performing Test C_HAS_WARNING-Wdisabled-optimization - Success #4 2.545 -- Performing Test C_HAS_WARNING-Wextra #4 2.625 -- Performing Test C_HAS_WARNING-Wextra - Success #4 2.626 -- Performing Test C_HAS_WARNING-Wformat_2 #4 2.702 -- Performing Test C_HAS_WARNING-Wformat_2 - Success #4 2.703 -- Performing Test C_HAS_WARNING-Winvalid-pch #4 2.778 -- Performing Test C_HAS_WARNING-Winvalid-pch - Success #4 2.779 -- Performing Test C_HAS_WARNING-Wno-format-nonliteral #4 2.854 -- Performing Test C_HAS_WARNING-Wno-format-nonliteral - Success #4 2.854 -- Performing Test C_HAS_WARNING-Wpointer-arith #4 2.930 -- Performing Test C_HAS_WARNING-Wpointer-arith - Success #4 2.931 -- Performing Test C_HAS_WARNING-Wshadow #4 3.007 -- Performing Test C_HAS_WARNING-Wshadow - Success #4 3.008 -- Performing Test C_HAS_WARNING-Wunused #4 3.082 -- Performing Test C_HAS_WARNING-Wunused - Success #4 3.083 -- Performing Test C_HAS_WARNING-Wwrite-strings #4 3.159 -- Performing Test C_HAS_WARNING-Wwrite-strings - Success #4 3.160 -- Performing Test C_HAS_WARNING-funit-at-a-time #4 3.236 -- Performing Test C_HAS_WARNING-funit-at-a-time - Success #4 3.237 -- Performing Test C_HAS_WARNING-Wno-strict-overflow #4 3.314 -- Performing Test C_HAS_WARNING-Wno-strict-overflow - Success #4 3.315 -- Performing Test CXX_HAS_WARNING-Wall #4 3.404 -- Performing Test CXX_HAS_WARNING-Wall - Success #4 3.405 -- Performing Test CXX_HAS_WARNING-Wno-long-double #4 3.496 -- Performing Test CXX_HAS_WARNING-Wno-long-double - Failed #4 3.497 -- Performing Test CXX_HAS_WARNING-Wcast-align #4 3.584 -- Performing Test CXX_HAS_WARNING-Wcast-align - Success #4 3.585 -- Performing Test CXX_HAS_WARNING-Wdisabled-optimization #4 3.674 -- Performing Test CXX_HAS_WARNING-Wdisabled-optimization - Success #4 3.675 -- Performing Test CXX_HAS_WARNING-Wextra #4 3.767 -- Performing Test CXX_HAS_WARNING-Wextra - Success #4 3.768 -- Performing Test CXX_HAS_WARNING-Wformat_2 #4 3.858 -- Performing Test CXX_HAS_WARNING-Wformat_2 - Success #4 3.859 -- Performing Test CXX_HAS_WARNING-Winvalid-pch #4 3.948 -- Performing Test CXX_HAS_WARNING-Winvalid-pch - Success #4 3.949 -- Performing Test CXX_HAS_WARNING-Wno-format-nonliteral #4 4.037 -- Performing Test CXX_HAS_WARNING-Wno-format-nonliteral - Success #4 4.038 -- Performing Test CXX_HAS_WARNING-Wpointer-arith #4 4.128 -- Performing Test CXX_HAS_WARNING-Wpointer-arith - Success #4 4.129 -- Performing Test CXX_HAS_WARNING-Wshadow #4 4.225 -- Performing Test CXX_HAS_WARNING-Wshadow - Success #4 4.225 -- Performing Test CXX_HAS_WARNING-Wunused #4 4.314 -- Performing Test CXX_HAS_WARNING-Wunused - Success #4 4.314 -- Performing Test CXX_HAS_WARNING-Wwrite-strings #4 4.403 -- Performing Test CXX_HAS_WARNING-Wwrite-strings - Success #4 4.404 -- Performing Test CXX_HAS_WARNING-funit-at-a-time #4 4.493 -- Performing Test CXX_HAS_WARNING-funit-at-a-time - Success #4 4.494 -- Performing Test CXX_HAS_WARNING-Wno-strict-overflow #4 4.586 -- Performing Test CXX_HAS_WARNING-Wno-strict-overflow - Success #4 4.587 -- Performing Test CXX_HAS_WARNING-Wno-deprecated #4 4.681 -- Performing Test CXX_HAS_WARNING-Wno-deprecated - Success #4 4.682 -- Performing Test CXX_HAS_WARNING-Wno-invalid-offsetof #4 4.778 -- Performing Test CXX_HAS_WARNING-Wno-invalid-offsetof - Success #4 4.779 -- Performing Test CXX_HAS_WARNING-Wno-undefined-var-template #4 4.869 -- Performing Test CXX_HAS_WARNING-Wno-undefined-var-template - Failed #4 4.870 -- Performing Test CXX_HAS_WARNING-Woverloaded-virtual #4 4.968 -- Performing Test CXX_HAS_WARNING-Woverloaded-virtual - Success #4 4.969 -- Performing Test CXX_HAS_WARNING-Wstrict-null-sentinel #4 5.076 -- Performing Test CXX_HAS_WARNING-Wstrict-null-sentinel - Success #4 5.077 -- Performing Test C_HAS_WARNING-mtune_native #4 5.159 -- Performing Test C_HAS_WARNING-mtune_native - Success #4 5.160 -- Performing Test C_HAS_WARNING-march_corei7 #4 5.242 -- Performing Test C_HAS_WARNING-march_corei7 - Success #4 5.243 -- Performing Test CXX_HAS_WARNING-mtune_native #4 5.331 -- Performing Test CXX_HAS_WARNING-mtune_native - Success #4 5.332 -- Performing Test CXX_HAS_WARNING-march_corei7 #4 5.428 -- Performing Test CXX_HAS_WARNING-march_corei7 - Success #4 5.428 -- Performing Test have_gold #4 5.506 -- Performing Test have_gold - Success #4 5.506 -- Performing Test CXX_HAS_DISABLE_OPTIMIZATION_FLAG #4 5.535 -- Performing Test CXX_HAS_DISABLE_OPTIMIZATION_FLAG - Failed #4 5.542 CMake Error at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message): #4 5.542 Could NOT find Git (missing: GIT_EXECUTABLE) #4 5.542 Call Stack (most recent call first): #4 5.542 /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE) #4 5.542 /usr/share/cmake-3.18/Modules/FindGit.cmake:96 (find_package_handle_standard_args) #4 5.542 SuperBuild.cmake:23 (find_package) #4 5.542 CMakeLists.txt:67 (include) #4 5.542 #4 5.542 #4 5.542 -- Configuring incomplete, errors occurred! #4 5.542 See also "/tmp/ants/build/CMakeFiles/CMakeOutput.log". #4 5.542 See also "/tmp/ants/build/CMakeFiles/CMakeError.log". #4 5.547 make: *** No targets specified and no makefile found. Stop. #4 5.548 /bin/sh: 1: cd: can't cd to ./ANTS-build ------ process "/bin/sh -c cd /tmp/ants/ && wget -q https://github.com/ANTsX/ANTs/archive/refs/tags/${ANTS_VERSION}.zip -O ants.zip && unzip ants.zip && rm ants.zip && cd /tmp/ants/build && cmake -DCMAKE_INSTALL_PREFIX=/opt/ants -DBUILD_SHARED_LIBS=OFF -DUSE_VTK=OFF -DSuperBuild_ANTS_USE_GIT_PROTOCOL=OFF -DBUILD_TESTING=OFF -DRUN_LONG_TESTS=OFF -DRUN_SHORT_TESTS=OFF /tmp/ants/ANTs-${ANTS_VERSION#v} 2>&1 | tee cmake.log && make -j 4 2>&1 | tee build.log && cd ./ANTS-build && make install 2>&1 | tee install.log && mv /tmp/ants/ANTs-${ANTS_VERSION#v}/Scripts/* /opt/ants/bin/ && rm -fr /tmp/ants" did not complete successfully: exit code: 2 ```

Notice the line that says:

#4 5.542   Could NOT find Git (missing: GIT_EXECUTABLE)

In any case, I will follow your advice and build it in stages, as in the official Dockerfile.

gdevenyi commented 1 year ago

Why can't you use the ANTs docker image as your base image? Or download and use the binaries provided?

cookpa commented 1 year ago

Thanks, I think this use case will not work because the Superbuild requires git to fetch ITK. To avoid git, it would be necessary to compile ITK separately. This is do-able but not trivial, and even then the ANTs cmake might need fixes to not require git.

Another option would be to copy binaries directly from the antsx/ants repository, with

COPY --from=antsx/ants:v2.4.2 /opt/ants /opt/ants

these binaries have shared libs on (so you'll want to set LD_LIBRARY_PATH), which reduces container size on disk.

The binaries on the Github releases page are built statically - unfortunately there isn't one for Ubuntu 18.04 for ANTs 2.4.2 becuase the native runner was deprecated, but I put it back for later versions using a docker build process.