search

tromp / cuckoo

a memory-bound graph-theoretic proof-of-work system
Other
818 stars 173 forks source link

Test building Linux CUDA miner on CI #66

Closed lucafavatella closed 4 years ago

lucafavatella commented 5 years ago

Could you rewrite the test to just check if CUDA is already available, and if not, skip the test?

@tromp I am not sure I get what you suggest. This is the CI configuration, so we know that CUDA toolkit is absent (no CUDA hardware or driver is needed in order to build the miner). So, if we want to have CI test building the CUDA miner, we need to configure CI to automatically download and install CUDA. There are two options:

I tried option B without success still getting nvcc fatal : Unknown option 'std'. If I read the CI logs correctly, with apt addon CI installs nvidia-cuda-toolkit_5.5.22-3ubuntu1_amd64.deb that seems old. Maybe there is better way (other apt repo?) though I am unable to spend further time on this now. Draft patch below.

Simplify CUDA toolkit installation

... at the expense of losing control of toolkit version.

References:

If only package nvidia-cuda-toolkit installed, CI fails with following error:

The following packages have unmet dependencies:
 nvidia-cuda-toolkit : Depends: nvidia-profiler (= 5.5.22-3ubuntu1) but it is not going to be installed
                       Depends: nvidia-cuda-dev (= 5.5.22-3ubuntu1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
apt-get install failed

Addresses https://github.com/tromp/cuckoo/pull/66#pullrequestreview-184146027

@@ -3,8 +3,8 @@ sudo: false
 language: cpp
 env:
   - JOB=cuckoo
-  - JOB=cuckoo-cuda MINER=cuda29  CUDA_DISTRO=ubuntu1404 CUDA_VSN_MAJOR=10 CUDA_VSN_MINOR=0 CUDA_VSN_PATCH=130-1 CUDA_ARCH=amd64 CUDA_PKG=cuda-toolkit
-  - JOB=cuckoo-cuda MINER=lcuda29 CUDA_DISTRO=ubuntu1404 CUDA_VSN_MAJOR=10 CUDA_VSN_MINOR=0 CUDA_VSN_PATCH=130-1 CUDA_ARCH=amd64 CUDA_PKG=cuda-toolkit
+  - JOB=cuckoo-cuda MINER=cuda29
+  - JOB=cuckoo-cuda MINER=lcuda29
   - JOB=cuckatoo
   - JOB=java
   - JOB=doc
@@ -16,26 +16,14 @@ matrix:
       env: JOB=java
 addons:
   apt:
+    update: true # For preventing error `nvcc fatal   : Unknown option 'std'`
     packages:
+      - nvidia-cuda-toolkit
+      - nvidia-cuda-dev # Needed by nvidia-cuda-toolkit
+      - nvidia-profiler # Needed by nvidia-cuda-toolkit
       - texlive-latex-base
       - texlive-pictures
       - pgf
-before_install:
-  - |
-    if test linux = "${TRAVIS_OS_NAME:?}" -a cuckoo-cuda = "${JOB:?}"; then
-      # Refs:
-      # * https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu-installation
-      # * https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#package-manager-metas
-      # * https://developer.nvidia.com/cuda-downloads
-      CUDA_VSN=${CUDA_VSN_MAJOR:?}.${CUDA_VSN_MINOR:?}.${CUDA_VSN_PATCH:?}
-      curl -OfsS https://developer.download.nvidia.com/compute/cuda/repos/${CUDA_DISTRO:?}/x86_64/cuda-repo-${CUDA_DISTRO:?}_${CUDA_VSN:?}_${CUDA_ARCH:?}.deb \
-      && sudo dpkg -i cuda-repo-${CUDA_DISTRO:?}_${CUDA_VSN:?}_${CUDA_ARCH:?}.deb \
-      && rm cuda-repo-${CUDA_DISTRO:?}_${CUDA_VSN:?}_${CUDA_ARCH:?}.deb \
-      && sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/${CUDA_DISTRO:?}/x86_64/7fa2af80.pub \
-      && sudo apt-get update \
-      && sudo apt-get install cuda-toolkit-${CUDA_VSN_MAJOR:?}-${CUDA_VSN_MINOR:?} \
-      && export PATH=/usr/local/cuda-${CUDA_VSN_MAJOR:?}.${CUDA_VSN_MINOR}/bin${PATH:+:${PATH}}
-    fi
 before_script:
   - case "${TRAVIS_OS_NAME:?}" in linux) LIBV=LD_LIBRARY_PATH;; osx) LIBV=DYLD_LIBRARY_PATH;; esac
   - echo "The library path variable name is ${LIBV:?}"
tromp commented 5 years ago

dear Luca,

There are two options:

(A) Download and install step by step as a user would. This is what this PR does. (B) Download and install relying on Travis CI apt addon.

I'd prefer to try to get option (B) working.

I tried option B without success still getting nvcc fatal : Unknown option 'std'.

Perhaps due to my recent Makefile change introducing NVCC ?= nvcc -std=c++11

This means that if you first define NVCC=nvcc it will avoid the -std option. Could you try to get that to work?

regards, -John

lucafavatella commented 5 years ago

I'd prefer to try to get option (B) working.

This means that if you first define NVCC=nvcc it will avoid the -std option.

Could you try to get that to work?

Attempt.

CI job failure:

$ if test cuckoo-cuda = "${JOB:?}"; then make -C src/cuckoo NVCC=nvcc ${MINER:?}; fi
make: Entering directory `/home/travis/build/aeternity/cuckoo/src/cuckoo'
nvcc -o cuda29 -DEDGEBITS=29 -arch sm_35 mean.cu ../crypto/blake2b-ref.c
In file included from /usr/include/c++/4.8/chrono:35:0,
                 from cuckoo.h:8,
                 from mean.cu:10:
/usr/include/c++/4.8/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support for the \
  ^
make: *** [cuda29] Error 1
make: Leaving directory `/home/travis/build/aeternity/cuckoo/src/cuckoo'
tromp commented 5 years ago

How can it complain about both

nvcc fatal : Unknown option 'std'

and

must be enabled with the -std=c++11 option

? I'm very confused...

tolbrino commented 5 years ago

@lucafavatella You could cache the CUDA folders for later builds, see https://docs.travis-ci.com/user/caching/

So if the folder doesn't exist, run through the full installation once. Then if it exists, skip the installation.

lucafavatella commented 4 years ago

I do not plan to update this PR so closing. Things missing: