rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.28k stars 884 forks source link

[BUG] "build.sh cudfjar" error: "cudfjni" links to target "kvikio::kvikio" was not found #10731

Closed gerashegalov closed 2 years ago

gerashegalov commented 2 years ago

Describe the bug When using build.sh cudfjar on HEAD we observe a build failure in the Maven cmake step.

Steps/Code to reproduce bug Execute from the top directory of the local git repo:

~/gits/rapidsai/cudf$ PARALLEL_LEVEL=8 ./build.sh clean cudfjar
build log details
[614/614] Linking CXX static library libcudftestutil.a
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building cudfjni 22.06.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
...
     [exec] CMake Error at CMakeLists.txt:125 (add_library):
     [exec]   Target "cudfjni" links to target "kvikio::kvikio" but the target was not
     [exec]   found.  Perhaps a find_package() call is missing for an IMPORTED target, or
     [exec]   an ALIAS target is missing?
...
     [exec] CMake Generate step failed.  Build files cannot be regenerated correctly.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.030s
[INFO] Finished at: Mon Apr 25 20:24:51 UTC 2022
[INFO] Final Memory: 16M/481M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8:run (cmake) on project cudf: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...... @ 5:90 in /rapids/cudf/java/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Expected behavior build.sh [clean] cudfjar should produce a portable cudf jar without issues

Environment overview (please complete the following information)

Environment details

$ nvidia-docker run -it -u 1000:1000 --rm -v /etc/group:/etc/group:ro -v /etc/passwd:/etc/passwd:ro -v /etc/shadow:/etc/shadow:ro -v /etc/sudoers.d:/etc/sudoers.d:ro -v ~/gits/rapidsai/cudf:/rapids/cudf:rw -v ~/.m2/repository:/rapids/.m2/repo
sitory:rw --workdir /rapids/cudf/java/target/libcudf-cmake-build cudf-build:11.5.0-devel-centos7 scl enable devtoolset-9 /rapids/cudf/print_env.sh
Click here to see environment details

     **git***
     commit 75f3873b87af8871c4fb85077df3dba4bbe24e2a (HEAD, origin/branch-22.06, branch-22.06)
     Author: Bradley Dice 
     Date:   Mon Apr 25 13:09:30 2022 -0500

     Use upstream clang-format pre-commit hook. (#10659)

     This PR improves the use of clang-format in pre-commit and CI by using a hook that is managed and cached by pre-commit. Developers who do not use pre-commit will see no changes to their workflow. Developers using pre-commit will be guaranteed that their code is formatted with the pinned version of clang-format (and will always align with CI's style check expectations) even if they have a different version of clang-format (or no version of clang-format) installed on their path. Like all pre-commit hooks for linting CMake and Python code, this version pinning will need to be kept in sync with the conda environment if we choose to upgrade clang-format.

     Note: this is aligned with how cuCollections uses clang-format, and it seems that has gone well. Thanks to @PointKernel for testing it on that repo!
     https://github.com/NVIDIA/cuCollections/blob/dev/.pre-commit-config.yaml

     Authors:
     - Bradley Dice (https://github.com/bdice)

     Approvers:
     - Yunsong Wang (https://github.com/PointKernel)
     - Vyas Ramasubramani (https://github.com/vyasr)

     URL: https://github.com/rapidsai/cudf/pull/10659
     **git submodules***
You need to run this command from the toplevel of the working tree.

     ***OS Information***
     CentOS Linux release 7.9.2009 (Core)
     NAME="CentOS Linux"
     VERSION="7 (Core)"
     ID="centos"
     ID_LIKE="rhel fedora"
     VERSION_ID="7"
     PRETTY_NAME="CentOS Linux 7 (Core)"
     ANSI_COLOR="0;31"
     CPE_NAME="cpe:/o:centos:centos:7"
     HOME_URL="https://www.centos.org/"
     BUG_REPORT_URL="https://bugs.centos.org/"

     CENTOS_MANTISBT_PROJECT="CentOS-7"
     CENTOS_MANTISBT_PROJECT_VERSION="7"
     REDHAT_SUPPORT_PRODUCT="centos"
     REDHAT_SUPPORT_PRODUCT_VERSION="7"

     CentOS Linux release 7.9.2009 (Core)
     CentOS Linux release 7.9.2009 (Core)
     Linux 73a60e229f11 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Mon Apr 25 20:41:50 2022
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 510.60.02    Driver Version: 512.15       CUDA Version: 11.6     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVIDIA RTX A300...  On   | 00000000:01:00.0 Off |                  N/A |
     | N/A   45C    P0    19W /  N/A |      0MiB /  6144MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |  No running processes found                                                 |
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:          x86_64
     CPU op-mode(s):        32-bit, 64-bit
     Byte Order:            Little Endian
     CPU(s):                16
     On-line CPU(s) list:   0-15
     Thread(s) per core:    2
     Core(s) per socket:    8
     Socket(s):             1
     Vendor ID:             GenuineIntel
     CPU family:            6
     Model:                 141
     Model name:            11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
     Stepping:              1
     CPU MHz:               2496.000
     BogoMIPS:              4992.00
     Hypervisor vendor:     Microsoft
     Virtualization type:   full
     L1d cache:             48K
     L1i cache:             32K
     L2 cache:              1280K
     L3 cache:              24576K
     Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid fsrm avx512_vp2intersect flush_l1d arch_capabilities

     ***CMake***
     /usr/local/cmake-3.22.3-linux-x86_64/bin/cmake
     cmake version 3.22.3

     CMake suite maintained and supported by Kitware (kitware.com/cmake).

     ***g++***
     /opt/rh/devtoolset-9/root/usr/bin/g++
     g++ (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2)
     Copyright (C) 2019 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /usr/local/cuda/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2021 NVIDIA Corporation
     Built on Mon_Sep_13_19:13:29_PDT_2021
     Cuda compilation tools, release 11.5, V11.5.50
     Build cuda_11.5.r11.5/compiler.30411180_0

     ***Python***
     /usr/bin/python
     Python 2.7.5

     ***Environment Variables***
     PATH                            : /opt/rh/devtoolset-9/root/usr/bin:/usr/local/cmake-3.22.3-linux-x86_64/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
     LD_LIBRARY_PATH                 : /opt/rh/devtoolset-9/root/usr/lib64:/opt/rh/devtoolset-9/root/usr/lib:/opt/rh/devtoolset-9/root/usr/lib64/dyninst:/opt/rh/devtoolset-9/root/usr/lib/dyninst:/opt/rh/devtoolset-9/root/usr/lib64:/opt/rh/devtoolset-9/root/usr/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    :
     PYTHON_PATH                     :

     conda not found
     pip not found

Additional context N/A

jlowe commented 2 years ago

This is a duplicate of #10700 and will be fixed by #10704.