search

browsermt / marian-dev

Fast Neural Machine Translation in C++ - development repository
https://marian-nmt.github.io
Other
20 stars 7 forks source link

Jenkins browsermt-marian-dev-cuda-9.2 #24 failed #31

Closed marianminion closed 2 years ago

marianminion commented 3 years ago

Build 'browsermt-marian-dev-cuda-9.2' is failing!

Last 50 lines of build output:

[...truncated 1.38 KB...]
Building on master in workspace /var/lib/jenkins/workspace/browsermt-marian-dev-cuda-9.2
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/browsermt/marian-dev # timeout=10
Fetching upstream changes from https://github.com/browsermt/marian-dev
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/browsermt/marian-dev +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision c24d0fd909d319f0a5380e4bcf83587ca3881386 (refs/remotes/origin/master)
Commit message: "Added wasm compile workflow for ubuntu"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f c24d0fd909d319f0a5380e4bcf83587ca3881386
 > git rev-list b08b26b5cfb9c6498ef48c1ee253f5bd5f788236 # timeout=10
[browsermt-marian-dev-cuda-9.2] $ /bin/sh -xe /tmp/jenkins8775765910282522168.sh
+ . /etc/environment
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
+ mkdir -p build
+ rm -rf build/CMakeCache.txt build/CMakeFiles build/CPackConfig.cmake build/CPackSourceConfig.cmake build/CTestTestfile.cmake build/Doxyfile build/Makefile build/Testing build/cmake_install.cmake build/iris_example build/libmarian.a build/local build/marian build/marian-conv build/marian-decoder build/marian-scorer build/marian-vocab build/mnist_example build/spm_decode build/spm_encode build/spm_export_vocab build/spm_normalize build/spm_train build/src build/test_cli build/test_dropout build/test_logger build/test_pooling build/test_prod build/test_sentencepiece_norm build/test_sqlite
+ cd build
+ cmake --version
cmake version 3.5.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).
+ cat /var/lib/jenkins/cuda-9.2/version.txt
CUDA Version 9.2.148
+ cmake -DCOMPILE_TESTS=ON -DCOMPILE_EXAMPLES=ON -DUSE_CUDNN=ON -DCUDA_TOOLKIT_ROOT_DIR=/var/lib/jenkins/cuda-9.2 ..
-- The CXX compiler identification is GNU 5.4.0
-- The C compiler identification is GNU 5.4.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
CMake Error at CMakeLists.txt:64 (add_compile_definitions):
  Unknown CMake command "add_compile_definitions".

-- Configuring incomplete, errors occurred!
See also "/var/lib/jenkins/workspace/browsermt-marian-dev-cuda-9.2/build/CMakeFiles/CMakeOutput.log".
Build step 'Execute shell' marked build as failure
Skipped archiving because build is not successful

Changes since last successful build:

View full output

abhi-agg commented 3 years ago

View full output

@jerinphilip Could you provide the complete logs? I can't open this link (gives me Not Found error)

jerinphilip commented 3 years ago

This is a cmake failure, from changing add_definitions to add_compile_definitions lol. The solution is to downgrade your cmake to something lower temporarily and bring the new stuff down. We have a very old cmake on jenkins. But the catch is this makes build configurations here compatible with upstream to be moved over there.

/cc @kpu

jerinphilip commented 3 years ago
Full build log

``` Started by upstream project "marian-dev-cuda-10.0" build number 199 originally caused by: Started by upstream project "marian-dev-cuda-10.1" build number 190 originally caused by: Started by GitHub push by emjotde Started by GitHub push by emjotde Running as SYSTEM Building on master in workspace /var/lib/jenkins/workspace/browsermt-marian-dev-cuda-9.2 > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/browsermt/marian-dev # timeout=10 Fetching upstream changes from https://github.com/browsermt/marian-dev > git --version # timeout=10 > git fetch --tags --progress https://github.com/browsermt/marian-dev +refs/heads/*:refs/remotes/origin/* > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10 Checking out Revision c24d0fd909d319f0a5380e4bcf83587ca3881386 (refs/remotes/origin/master) Commit message: "Added wasm compile workflow for ubuntu" > git config core.sparsecheckout # timeout=10 > git checkout -f c24d0fd909d319f0a5380e4bcf83587ca3881386 > git rev-list b08b26b5cfb9c6498ef48c1ee253f5bd5f788236 # timeout=10 [browsermt-marian-dev-cuda-9.2] $ /bin/sh -xe /tmp/jenkins8775765910282522168.sh + . /etc/environment + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games + mkdir -p build + rm -rf build/CMakeCache.txt build/CMakeFiles build/CPackConfig.cmake build/CPackSourceConfig.cmake build/CTestTestfile.cmake build/Doxyfile build/Makefile build/Testing build/cmake_install.cmake build/iris_example build/libmarian.a build/local build/marian build/marian-conv build/marian-decoder build/marian-scorer build/marian-vocab build/mnist_example build/spm_decode build/spm_encode build/spm_export_vocab build/spm_normalize build/spm_train build/src build/test_cli build/test_dropout build/test_logger build/test_pooling build/test_prod build/test_sentencepiece_norm build/test_sqlite + cd build + cmake --version cmake version 3.5.1 CMake suite maintained and supported by Kitware (kitware.com/cmake). + cat /var/lib/jenkins/cuda-9.2/version.txt CUDA Version 9.2.148 + cmake -DCOMPILE_TESTS=ON -DCOMPILE_EXAMPLES=ON -DUSE_CUDNN=ON -DCUDA_TOOLKIT_ROOT_DIR=/var/lib/jenkins/cuda-9.2 .. -- The CXX compiler identification is GNU 5.4.0 -- The C compiler identification is GNU 5.4.0 -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done CMake Error at CMakeLists.txt:64 (add_compile_definitions): Unknown CMake command "add_compile_definitions". -- Configuring incomplete, errors occurred! See also "/var/lib/jenkins/workspace/browsermt-marian-dev-cuda-9.2/build/CMakeFiles/CMakeOutput.log". Build step 'Execute shell' marked build as failure Skipped archiving because build is not successful GitHub Issue Notifier: Build has started failing, filed GitHub issue #31 Finished: FAILURE ```

abhi-agg commented 3 years ago

We have a very old cmake on jenkins.

It means cmake version installed on Jenkins is 2.x.x (which is very old) because add_compile_definition is supported since cmake version 3.0.0.

But the catch is this makes build configurations here compatible with upstream to be moved over there.

I didn't understand this one. Which build configurations are you talking about?

Rather than downgrading from add_compile_definition to add_definition, wouldn't it make more sense to upgrade cmake package on the jenkins system to be consistent with what marian actually requires (a minimum cmake version 3.5.1)?

All that is needed is to update the cmake package on Jenkins.

jerinphilip commented 3 years ago

Upstream doesn't have a single add_compile_definitions [ref]. Also, a good enough reason - consistency. I guess the lower CMake in a way enforces it. The edits should leave Jenkins build config in a consistent state (similar for upstream and bergamot) as well, so I'm not in favour of your recommendation.

The submitter of a PR is responsible to keep all CI checks successful. These workflows are added for a reason (I already explained).

abhi-agg commented 3 years ago

The submitter of a PR is responsible to keep all CI checks successful. These workflows are added for a reason (I already explained).

CI checks are indeed successful. It is the Jenkins testers that are inconsistent with what marian expects. I am fine with making the change as well. Looking for the right call here.

@kpu My proposition is already in my previous comment. Please let me know your final call on this (I am fine with downgrading cmake commands but just want to do the right thing here) 👍

kpu commented 3 years ago

We can upgrade cmake.

abhi-agg commented 3 years ago

Thanks. @jerinphilip Would you be doing the cmake upgrade?

jerinphilip commented 2 years ago

cmake looks upgraded.