Cannot compile

[Marigold]

The image stopped working a few days ago due to erorr in XGBoost compilation

Step 3/4 : RUN apk add --update --no-cache     --virtual=.build-dependencies     make gfortran     python3-dev     py-setuptools g++ &&     apk add --no-cache openblas lapack-dev libexecinfo-dev libstdc++ libgomp &&     pip install numpy==1.13.3 &&     pip install scipy==1.0.0 &&     pip install pandas==0.22.0 scikit-learn==0.19.1 &&     ln -s locale.h /usr/include/xlocale.h &&     cd /src/xgboost; make -j4 &&     cd /src/xgboost/python-package &&     python3 setup.py install &&     rm /usr/include/xlocale.h &&     rm -r /root/.cache &&     rm -rf /src &&     apk del .build-dependencies
 ---> Running in 1c441ba2d695
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/24) Installing make (4.2.1-r0)
(2/24) Installing binutils-libs (2.30-r1)
(3/24) Installing binutils (2.30-r1)
(4/24) Installing gmp (6.1.2-r0)
(5/24) Installing isl (0.17.1-r0)
(6/24) Installing libgomp (6.3.0-r4)
(7/24) Installing libatomic (6.3.0-r4)
(8/24) Installing pkgconf (1.3.7-r0)
(9/24) Installing libgcc (6.3.0-r4)
(10/24) Installing mpfr3 (3.1.5-r0)
(11/24) Installing mpc1 (1.0.3-r0)
(12/24) Installing libstdc++ (6.3.0-r4)
(13/24) Installing gcc (6.3.0-r4)
(14/24) Installing libquadmath (6.3.0-r4)
(15/24) Installing libgfortran (6.3.0-r4)
(16/24) Installing gfortran (6.3.0-r4)
(17/24) Installing python3 (3.6.5-r0)
(18/24) Installing python3-dev (3.6.5-r0)
(19/24) Installing python2 (2.7.15-r0)
(20/24) Installing py-setuptools (33.1.1-r1)
(21/24) Installing musl-dev (1.1.16-r14)
(22/24) Installing libc-dev (0.7.1-r0)
(23/24) Installing g++ (6.3.0-r4)
(24/24) Installing .build-dependencies (0)
Executing busybox-1.26.2-r9.trigger
OK: 321 MiB in 59 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/5) Installing lapack (3.7.0-r0)
(2/5) Installing lapack-dev (3.7.0-r0)
(3/5) Installing libexecinfo (1.1-r0)
(4/5) Installing libexecinfo-dev (1.1-r0)
(5/5) Installing openblas (0.2.19-r3)
Executing busybox-1.26.2-r9.trigger
OK: 366 MiB in 64 packages
Collecting numpy==1.13.3
  Downloading https://files.pythonhosted.org/packages/bf/2d/005e45738ab07a26e621c9c12dc97381f372e06678adf7dc3356a69b5960/numpy-1.13.3.zip (5.0MB)
Building wheels for collected packages: numpy
  Running setup.py bdist_wheel for numpy: started
  Running setup.py bdist_wheel for numpy: still running...
  Running setup.py bdist_wheel for numpy: still running...
  Running setup.py bdist_wheel for numpy: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/b6/10/65/189b772e73b4505109d5a1e6671b07e65797023718777295e0
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.13.3
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting scipy==1.0.0
  Downloading https://files.pythonhosted.org/packages/d0/73/76fc6ea21818eed0de8dd38e1e9586725578864169a2b31acdeffb9131c8/scipy-1.0.0.tar.gz (15.2MB)
Building wheels for collected packages: scipy
  Running setup.py bdist_wheel for scipy: started
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: still running...
  Running setup.py bdist_wheel for scipy: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/38/d4/55/5d300ffc87fc5297a63778cd504312ff957b95b897780a2b1b
Successfully built scipy
Installing collected packages: scipy
Successfully installed scipy-1.0.0
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pandas==0.22.0
  Downloading https://files.pythonhosted.org/packages/08/01/803834bc8a4e708aedebb133095a88a4dad9f45bbaf5ad777d2bea543c7e/pandas-0.22.0.tar.gz (11.3MB)
Collecting scikit-learn==0.19.1
  Downloading https://files.pythonhosted.org/packages/f5/2c/5edf2488897cad4fb8c4ace86369833552615bf264460ae4ef6e1f258982/scikit-learn-0.19.1.tar.gz (9.5MB)
Collecting python-dateutil>=2 (from pandas==0.22.0)
  Downloading https://files.pythonhosted.org/packages/74/68/d87d9b36af36f44254a8d512cbfc48369103a3b9e474be9bdfe536abfc45/python_dateutil-2.7.5-py2.py3-none-any.whl (225kB)
Collecting pytz>=2011k (from pandas==0.22.0)
  Downloading https://files.pythonhosted.org/packages/f8/0e/2365ddc010afb3d79147f1dd544e5ee24bf4ece58ab99b16fbb465ce6dc0/pytz-2018.7-py2.py3-none-any.whl (506kB)
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python3.6/site-packages (from pandas==0.22.0)
Collecting six>=1.5 (from python-dateutil>=2->pandas==0.22.0)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Building wheels for collected packages: pandas, scikit-learn
  Running setup.py bdist_wheel for pandas: started
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/21/6e/f3/24ca4e2d50a98d3733ca82c0e0941d3b411b3212aacf7d9a73
  Running setup.py bdist_wheel for scikit-learn: started
  Running setup.py bdist_wheel for scikit-learn: still running...
  Running setup.py bdist_wheel for scikit-learn: still running...
  Running setup.py bdist_wheel for scikit-learn: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/5e/b1/d3/9023541e7435544183dd51d00e9ca58a38baef49cf51607b3a
Successfully built pandas scikit-learn
Installing collected packages: six, python-dateutil, pytz, pandas, scikit-learn
Successfully installed pandas-0.22.0 python-dateutil-2.7.5 pytz-2018.7 scikit-learn-0.19.1 six-1.11.0
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Makefile:31: MAKE [make] - checked OK
g++ -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
g++ -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
g++ -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/objective/regression_obj.o src/objective/regression_obj.cc >build/objective/regression_obj.d
g++ -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/objective/rank_obj.o src/objective/rank_obj.cc >build/objective/rank_obj.d
In file included from dmlc-core/include/dmlc/./serializer.h:22:0,
                 from dmlc-core/include/dmlc/io.h:445,
                 from src/learner.cc:7:
dmlc-core/include/dmlc/././endian.h:24:6: error: #error "Unable to determine endianness of your machine; use CMake to compile"
     #error "Unable to determine endianness of your machine; use CMake to compile"
      ^~~~~
In file included from dmlc-core/include/dmlc/././serializer.h:22:0,
                 from dmlc-core/include/dmlc/./io.h:445,
                 from dmlc-core/include/dmlc/data.h:14,
                 from include/xgboost/./data.h:11,
                 from include/xgboost/objective.h:15,
                 from src/objective/regression_obj.cu:10,
                 from src/objective/regression_obj.cc:17:
dmlc-core/include/dmlc/./././endian.h:24:6: error: #error "Unable to determine endianness of your machine; use CMake to compile"
     #error "Unable to determine endianness of your machine; use CMake to compile"
      ^~~~~
make: *** [Makefile:141: build/objective/regression_obj.o] Error 1
make: *** Waiting for unfinished jobs....
g++ -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/logging.cc -o build/logging.o
make: *** [Makefile:143: build/learner.o] Error 1
In file included from dmlc-core/include/dmlc/././serializer.h:22:0,
                 from dmlc-core/include/dmlc/./io.h:445,
                 from dmlc-core/include/dmlc/data.h:14,
                 from include/xgboost/./data.h:11,
                 from include/xgboost/objective.h:15,
                 from src/objective/rank_obj.cc:9:
dmlc-core/include/dmlc/./././endian.h:24:6: error: #error "Unable to determine endianness of your machine; use CMake to compile"
     #error "Unable to determine endianness of your machine; use CMake to compile"
      ^~~~~
make: *** [Makefile:143: build/objective/rank_obj.o] Error 1
ERROR: Service 'app' failed to build: The command '/bin/sh -c apk add --update --no-cache     --virtual=.build-dependencies     make gfortran     python3-dev     py-setuptools g++ &&     apk add --no-cache openblas lapack-dev libexecinfo-dev libstdc++ libgomp &&     pip install numpy==1.13.3 &&     pip install scipy==1.0.0 &&     pip install pandas==0.22.0 scikit-learn==0.19.1 &&     ln -s locale.h /usr/include/xlocale.h &&     cd /src/xgboost; make -j4 &&     cd /src/xgboost/python-package &&     python3 setup.py install &&     rm /usr/include/xlocale.h &&     rm -r /root/.cache &&     rm -rf /src &&     apk del .build-dependencies' returned a non-zero code: 2

I didn't find any solution yet...

[weelng]

Hi, I couldn't figure out where the root of the issue was with respect to compiling xgboost. However, if you require a quick & immediate fix for your project, you may want to edit this line: https://github.com/Nexosis/alpine-xgboost/blob/master/Dockerfile#L8 to instead reading the following: git clone --recursive -b v0.81 https://github.com/dmlc/xgboost && \ What it does is to refer back to the earlier stable branch, v0.81, instead of using the master branch of xgboost. Hope that helps!

[Marigold]

Thanks @weelng, that helped. I was trying using older versions of xgboost, but couldn't make it work for some reason. Your solution works like a charm though.