search

google / llvm-premerge-checks

CI system for premerge-testing in LLVM project
Apache License 2.0
41 stars 37 forks source link

libomptarget tests #368

Open Meinersbur opened 2 years ago

Meinersbur commented 2 years ago

Could we add libomptarget tests to the pre-merge checks? This would consist of 3 parts:

  1. Compile libomptarget and plugins
  2. Run host-offloading regression tests (use the CPU as offloading-target)
  3. Run device-offloading tests (CUDA and/or AMD)

Compiling libomptarget has some additional dependencies: libelf-dev, libffi-dev, gcc-multilib (Ubuntu/Debian package names) and the CUDA toolkit for device offloading. Could those be installed on the workers? Currently building libomptarget plugins are skipped by the pre-merge check workers with this cmake messages:

-- Could NOT find LIBOMPTARGET_DEP_LIBELF (missing: LIBOMPTARGET_DEP_LIBELF_LIBRARIES LIBOMPTARGET_DEP_LIBELF_INCLUDE_DIRS) 
-- Found LIBOMPTARGET_DEP_LIBFFI: /usr/lib/x86_64-linux-gnu/libffi.so  
-- Could NOT find LIBOMPTARGET_DEP_CUDA_DRIVER (missing: LIBOMPTARGET_DEP_CUDA_DRIVER_LIBRARIES) 
-- LIBOMPTARGET: Building offloading runtime library libomptarget.
-- LIBOMPTARGET: Not building AMDGPU plugin: LIBELF not found
-- LIBOMPTARGET: Not building CUDA offloading plugin: libelf dependency not found.
-- LIBOMPTARGET: Not building x86_64 offloading plugin: libelf dependency not found.

Once these dependencies are available, check-openmp should run the host-offloading regression tests. libomp(target) regressions tests are end-to-end tests that execute the compiled code. For device-offloading, it obviously need actual hardware (either NVIDIA or AMD) in the worker. @jdoerfert mentioned the LLVM project has GCE cloud credits we might be able to use for that, but even only running host-offloading tests would be helpful.

metaflow commented 2 years ago

I am planning to add this deps to agents. hat I see now (locally) with new deps:

-- LIBOMPTARGET: Building offloading runtime library libomptarget.
-- LIBOMPTARGET: Not building aarch64 offloading plugin: machine not found in the system.
-- LIBOMPTARGET: Building AMDGPU plugin for dlopened libhsa
-- LIBOMPTARGET: Not generating amdgcn test targets as amdgpu-arch is not found
-- LIBOMPTARGET: Building CUDA offloading plugin.
-- LIBOMPTARGET: Building CUDA plugin for dlopened libcuda
-- LIBOMPTARGET: Disabling tests using CUDA plugin as cuda may not be available
-- LIBOMPTARGET: Not building PPC64 offloading plugin: machine not found in the system.
-- LIBOMPTARGET: Not building PPC64le offloading plugin: machine not found in the system.
-- LIBOMPTARGET: Not building nec-aurora plugin: libveo or libveosinfo not found.
-- LIBOMPTARGET: Building x86_64 offloading plugin.
-- LIBOMPTARGET: Building AMDGCN device RTL. Using clang from in-tree build
-- LIBOMPTARGET: Building CUDA LLVM bitcode offloading device RTL using in-tree clang.
-- LIBOMPTARGET: Building DeviceRTL. Using clang from in-tree build
-- LIBOMPTARGET: Building the llvm-omp-device-info tool

Running ninja check-openmp fails 13 tests as "./bin/not" not found (I guess it should be in deps somewhere). Running after ninja not passes 379 tests for me.

@Meinersbur does this look good to you?

Meinersbur commented 2 years ago

The cmake output looks as expected (with no CUDA toolkit or ROCm installed), thanks. Is openmp configured with LLVM_ENABLE_PROJECTS=openmp or LLVM_ENABLE_RUNTIMES=openmp (both should work).

I'll have a look at the not dependency. Should be part of the normal test dependency, and I have not encountered it with my buildbots (eg. http://meinersbur.de:8011/#/builders/11)

Meinersbur commented 2 years ago

not fix here: https://reviews.llvm.org/D115454

metaflow commented 2 years ago

I have build an image and run tests on "clang;openmp" . Current setup : https://buildkite.com/llvm-project/llvm-main/builds/3881#96d288c5-b132-4b45-9fa4-aecac023e845

  | Skipped          :    11
  | Unsupported      :    75
  | Passed           : 29431
  | Expectedly Failed:    29

new: https://buildkite.com/llvm-project/llvm-main/builds/3880#e497f1f2-b098-48ac-8abf-cfe55d44760b

Skipped          :    11
  | Unsupported      :    75
  | Passed           : 29431
  | Expectedly Failed:    29

so I don't see any new test, guess we do care about compiling the offloading targets.

I have also updated toolchain in image to 13. Will update all agents today.

metaflow commented 2 years ago

@Meinersbur please tell if you need anything else. E.g. help with setup new agents with GPU

Meinersbur commented 2 years ago

I see new tests in the log file, e.g. PASS: libomptarget :: x86_64-pc-linux-gnu :: mapping/target_implicit_partial_map.c (364 of 401) that are not in the "current setup" configuration. I think you only looked at the summary report for check-clang ("Running the Clang regression tests"), but check-openmp has its own summary report:

Old:

Testing Time: 28.36s
  Unsupported:  33
  Passed     : 284

New:

Testing Time: 33.86s
  Unsupported:  50
  Passed     : 351

I think instead of ninja check-clang check-openmp, running ninja check gives just one summary report.

Meinersbur commented 2 years ago

@Meinersbur please tell if you need anything else. E.g. help with setup new agents with GPU

Running GPU tests with an actual GPU would be nice, if feasible.