search

TRON-Bioinformatics / neofox

Annotation of mutated peptide sequences with published or novel potential neoantigen descriptors
GNU General Public License v3.0
25 stars 6 forks source link

Problem building the Dockerfile #2

Closed ibrahimkurt closed 1 year ago

ibrahimkurt commented 1 year ago

Hi all!

I am trying to build a docker image using the Dockerfile on my M1 pro Arm64 but getting a series of errors. I resolved some of the issues by adding lines (marked w/ asterisk) shown below:

# build and install neofox package
RUN python3 setup.py bdist_wheel
*RUN apt-get -y update && apt-get -y upgrade
*RUN apt-get install -y gcc python3-dev
*RUN pip3 install numpy
*RUN pip3 install scipy  
*RUN pip3 install Cython
RUN pip3 install dist/*.whl

But couldn't manage to bypass the last error I am getting now as shown below:

#23 63.74       running build_clib
#23 63.74       customize UnixCCompiler
#23 63.74       customize UnixCCompiler using build_clib
#23 63.74       CCompilerOpt.cc_test_flags[1013] : testing flags (-march=native)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       creating /tmp/tmpa7mwrw8s/usr
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local/lib
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local/lib/python3.7
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local/lib/python3.7/site-packages
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local/lib/python3.7/site-packages/numpy
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local/lib/python3.7/site-packages/numpy/distutils
#23 63.74       creating /tmp/tmpa7mwrw8s/usr/local/lib/python3.7/site-packages/numpy/distutils/checks
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=native'
#23 63.74       CCompilerOpt.cc_test_flags[1013] : testing flags (-O3)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-O3'
#23 63.74       CCompilerOpt.cc_test_flags[1013] : testing flags (-Werror)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-Werror'
#23 63.74       CCompilerOpt.__init__[1701] : check requested baseline
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'ASIMD' with flags ()
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-Werror'
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'NEON' with flags ()
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-Werror'
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'NEON_VFPV4' with flags ()
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-Werror'
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'NEON_FP16' with flags ()
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-Werror'
#23 63.74       CCompilerOpt.__init__[1710] : check requested dispatch-able features
#23 63.74       CCompilerOpt.cc_test_flags[1013] : testing flags (-march=armv8.2-a+dotprod)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=armv8.2-a+dotprod'
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'ASIMDDP' with flags (-march=armv8.2-a+dotprod)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=armv8.2-a+dotprod -Werror'
#23 63.74       CCompilerOpt.cc_test_flags[1013] : testing flags (-march=armv8.2-a+fp16)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=armv8.2-a+fp16'
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'ASIMDHP' with flags (-march=armv8.2-a+fp16)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=armv8.2-a+fp16 -Werror'
#23 63.74       CCompilerOpt.cc_test_flags[1013] : testing flags (-march=armv8.2-a+fp16fml)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=armv8.2-a+fp16fml'
#23 63.74       CCompilerOpt.feature_test[1468] : testing feature 'ASIMDFHM' with flags (-march=armv8.2-a+fp16+fp16fml)
#23 63.74       C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       compile options: '-c'
#23 63.74       extra options: '-march=armv8.2-a+fp16+fp16fml -Werror'
#23 63.74       CCompilerOpt.dist_test[581] : CCompilerOpt._dist_test_spawn[718] : Command (gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -c /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c -o /tmp/tmpa7mwrw8s/usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.o -MMD -MF /tmp/tmpa7mwrw8s/usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.o.d -march=armv8.2-a+fp16+fp16fml -Werror) failed with exit status 1 output ->
#23 63.74       /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c: In function ‘main’:
#23 63.74       /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c:13:35: error: implicit declaration of function ‘vfmlal_low_u32’; did you mean ‘vfmlal_low_f16’? [-Werror=implicit-function-declaration]
#23 63.74          13 |     int ret  = (int)vget_lane_f32(vfmlal_low_u32(vlf, vlhp, vlhp), 0);
#23 63.74             |                                   ^~~~~~~~~~~~~~
#23 63.74             |                                   vfmlal_low_f16
#23 63.74       /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c:13:35: error: incompatible type for argument 1 of ‘vget_lane_f32’
#23 63.74          13 |     int ret  = (int)vget_lane_f32(vfmlal_low_u32(vlf, vlhp, vlhp), 0);
#23 63.74             |                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#23 63.74             |                                   |
#23 63.74             |                                   int
#23 63.74       In file included from /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c:4:
#23 63.74       /usr/lib/gcc/aarch64-linux-gnu/10/include/arm_neon.h:3169:28: note: expected ‘float32x2_t’ but argument is of type ‘int’
#23 63.74        3169 | vget_lane_f32 (float32x2_t __a, const int __b)
#23 63.74             |                ~~~~~~~~~~~~^~~
#23 63.74       /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c:14:36: error: implicit declaration of function ‘vfmlslq_high_u32’; did you mean ‘vmlsl_high_u32’? [-Werror=implicit-function-declaration]
#23 63.74          14 |         ret += (int)vgetq_lane_f32(vfmlslq_high_u32(vf, vhp, vhp), 0);
#23 63.74             |                                    ^~~~~~~~~~~~~~~~
#23 63.74             |                                    vmlsl_high_u32
#23 63.74       /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c:14:36: error: incompatible type for argument 1 of ‘vgetq_lane_f32’
#23 63.74          14 |         ret += (int)vgetq_lane_f32(vfmlslq_high_u32(vf, vhp, vhp), 0);
#23 63.74             |                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#23 63.74             |                                    |
#23 63.74             |                                    int
#23 63.74       In file included from /usr/local/lib/python3.7/site-packages/numpy/distutils/checks/cpu_asimdfhm.c:4:
#23 63.74       /usr/lib/gcc/aarch64-linux-gnu/10/include/arm_neon.h:3269:29: note: expected ‘float32x4_t’ but argument is of type ‘int’
#23 63.74        3269 | vgetq_lane_f32 (float32x4_t __a, const int __b)
#23 63.74             |                 ~~~~~~~~~~~~^~~
#23 63.74       cc1: all warnings being treated as errors
#23 63.74       
#23 63.74       CCompilerOpt.feature_test[1482] : testing failed
#23 63.74       CCompilerOpt.__init__[1723] : skip features (NEON_FP16 ASIMD NEON NEON_VFPV4) since its part of baseline
#23 63.74       CCompilerOpt.__init__[1726] : initialize targets groups
#23 63.74       CCompilerOpt.__init__[1728] : parse target group simd_test
#23 63.74       CCompilerOpt._parse_target_tokens[1941] : skip targets (SSE2 VSX VSX2 XOP (FMA3 AVX2) FMA4 AVX512F VSX3 AVX512_SKX SSE42) not part of baseline or dispatch-able features
#23 63.74       CCompilerOpt._parse_policy_not_keepbase[2051] : skip baseline features (ASIMD)
#23 63.74       CCompilerOpt.generate_dispatch_header[2272] : generate CPU dispatch header: (build/src.linux-aarch64-3.7/numpy/distutils/include/npy_cpu_dispatch_config.h)
#23 63.74       CCompilerOpt.generate_dispatch_header[2283] : dispatch header dir build/src.linux-aarch64-3.7/numpy/distutils/include does not exist, creating it
#23 63.74       building 'libsvm-skl' library
#23 63.74       compiling C++ sources
#23 63.74       C compiler: g++ -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#23 63.74       
#23 63.74       creating build/temp.linux-aarch64-3.7
#23 63.74       creating build/temp.linux-aarch64-3.7/sklearn
#23 63.74       creating build/temp.linux-aarch64-3.7/sklearn/svm
#23 63.74       creating build/temp.linux-aarch64-3.7/sklearn/svm/src
#23 63.74       creating build/temp.linux-aarch64-3.7/sklearn/svm/src/libsvm
#23 63.74       compile options: '-I/usr/local/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.linux-aarch64-3.7/numpy/distutils/include -c'
#23 63.74       g++: sklearn/svm/src/libsvm/libsvm_template.cpp
#23 63.74       
#23 63.74       
#23 63.74       [Errno 2] No such file or directory: 'g++': 'g++'
#23 63.74       
#23 63.74       
#23 63.74       error: Command "g++ -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.linux-aarch64-3.7/numpy/distutils/include -c sklearn/svm/src/libsvm/libsvm_template.cpp -o build/temp.linux-aarch64-3.7/sklearn/svm/src/libsvm/libsvm_template.o -MMD -MF build/temp.linux-aarch64-3.7/sklearn/svm/src/libsvm/libsvm_template.o.d" failed with exit status 127
#23 63.74       
#23 63.74       ########### CLIB COMPILER OPTIMIZATION ###########
#23 63.74       Platform      :
#23 63.74         Architecture: aarch64
#23 63.74         Compiler    : gcc
#23 63.74       
#23 63.74       CPU baseline  :
#23 63.74         Requested   : 'min'
#23 63.74         Enabled     : NEON NEON_FP16 NEON_VFPV4 ASIMD
#23 63.74         Flags       : none
#23 63.74         Extra checks: none
#23 63.74       
#23 63.74       CPU dispatch  :
#23 63.74         Requested   : 'max -xop -fma4'
#23 63.74         Enabled     : ASIMDHP ASIMDDP
#23 63.74         Generated   : none
#23 63.74       CCompilerOpt.cache_flush[809] : write cache to path -> /tmp/pip-install-fjgggoh4/scikit-learn_8dba9d22d83746e6aff508ae90f2745b/build/temp.linux-aarch64-3.7/ccompiler_opt_cache_clib.py
#23 63.74       [end of output]
#23 63.74   
#23 63.74   note: This error originates from a subprocess, and is likely not a problem with pip.
#23 63.74   WARNING: No metadata found in /usr/local/lib/python3.7/site-packages
#23 63.74   Rolling back uninstall of scikit-learn
#23 63.74   Moving to /usr/local/lib/python3.7/site-packages/scikit_learn-1.0.2.dist-info/
#23 63.74    from /usr/local/lib/python3.7/site-packages/~cikit_learn-1.0.2.dist-info
#23 63.74   Moving to /usr/local/lib/python3.7/site-packages/scikit_learn.libs/
#23 63.74    from /usr/local/lib/python3.7/site-packages/~cikit_learn.libs
#23 63.74   Moving to /usr/local/lib/python3.7/site-packages/sklearn/
#23 63.74    from /usr/local/lib/python3.7/site-packages/~klearn
#23 63.74 error: legacy-install-failure
#23 63.74 
#23 63.74 × Encountered error while trying to install package.
#23 63.74 ╰─> scikit-learn
#23 63.74 
#23 63.74 note: This is an issue with the package mentioned above, not pip.
#23 63.74 hint: See above for output from the failure.
#23 63.74 WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
#23 63.74 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c pip3 install dist/*.whl]: exit code: 1

I would really appreciate it if you could share some ideas with this issue. Thanks a lot!

ibrahimkurt commented 1 year ago

also FYI the new link for NetMHCIIpan-4.0 dataset is https://services.healthtech.dtu.dk/services/NetMHCIIpan-4.0/data.tar.gz

ibrahimkurt commented 1 year ago

Turns out the problem was that running this on an M1 macbook, Docker was running a native Arm image of ubuntu, instead of the emulated x86 image. For now the problem seems to be solved by running:

docker build --platform linux/x86_64 --tag neofox-docker .

instead of

docker build --tag neofox-docker .

priesgo commented 1 year ago

Thanks @ibrahimkurt , let us know if you find further trouble. I will make sure to update the link you refer to