llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.06k stars 11.98k forks source link

[Libclc] Please add support for AMD Renoir (gfx909 ? ) #52665

Closed sviscapi closed 9 months ago

sviscapi commented 2 years ago

Dear all,

My OpenCL tasks within BOINC all fail with the following error message:

OpenCL compiling FAILED! : -11 . Error message: fatal error: cannot open
file '/usr/lib/clc/gfx909-amdgcn-mesa-mesa3d.bc': No such file or directory

OpenCL device has no FP64 support
OpenCL compiling FAILED! : -11 . Error message: fatal error: cannot open
file '/usr/lib/clc/gfx909-amdgcn-mesa-mesa3d.bc': No such file or directory

The file "gfx909-amdgcn-mesa-mesa3d.bc" is indeed missing from the libclc-amdgcn package in my Linux distribution of choice (Debian 11 "Bullseye"):

https://packages.debian.org/bullseye/libclc-amdgcn

My setup sports an AMD Ryzen 7 4800H CPU with Radeon Graphics (AMD Renoir).

Do you think support for this recent GPU could be added in a future release ?

Best regards,

Samuel

andreish commented 2 years ago

same for amd ryzen7 5800 .

kayshinonome commented 2 years ago

I have this issue for the AMD Ryzen 3 3250

llvmbot commented 2 years ago

@llvm/issue-subscribers-opencl

llvmbot commented 2 years ago

@llvm/issue-subscribers-backend-amdgpu

sviscapi commented 2 years ago

Dear

Would the patch discussed in that issue https://github.com/llvm/llvm-project/issues/44186 also work in this case ?

Best regards,

Samuel

AnastasiaStulova commented 2 years ago

@tstellar, we don't seem to have a label for libclc, would it make sense to create one?

Mek101 commented 1 year ago

Same with Ryzen 3 5300U. Or is there any blocking issue?

cezarlamann commented 1 year ago

Hello everyone! I'm a noob using LLVM for anything, and I'd like to use CLBlast with my AMD Renoir APU (R7-4700U GFX 909 GCN 5.1), which requires LLVM to support gfx909. What can we do to get this working?

peci1 commented 1 year ago

After trying the trick with symlinking the missing .bc file, clinfo is satisfied but CL actually doesn't work.

clinfo ``` $ clinfo Number of platforms 1 Platform Name Clover Platform Vendor Mesa Platform Version OpenCL 1.1 Mesa 22.3.7 - kisak-mesa PPA Platform Profile FULL_PROFILE Platform Extensions cl_khr_icd Platform Extensions function suffix MESA Platform Name Clover Number of devices 1 Device Name RENOIR (renoir, LLVM 15.0.7, DRM 3.54, 6.5.3-060503-generic) Device Vendor AMD Device Vendor ID 0x1002 Device Version OpenCL 1.1 Mesa 22.3.7 - kisak-mesa PPA Driver Version 22.3.7 - kisak-mesa PPA Device OpenCL C Version OpenCL C 1.1 Device Type GPU Device Profile FULL_PROFILE Device Available Yes Compiler Available Yes Max compute units 7 Max clock frequency 1600MHz Max work item dimensions 3 Max work item sizes 256x256x256 Max work group size 256 Preferred work group size multiple 64 Preferred / native vector sizes char 16 / 16 short 8 / 8 int 4 / 4 long 2 / 2 half 0 / 0 (n/a) float 4 / 4 double 2 / 2 (cl_khr_fp64) Half-precision Floating-point support (n/a) Single-precision Floating-point support (core) Denormals No Infinity and NANs Yes Round to nearest Yes Round to zero No Round to infinity No IEEE754-2008 fused multiply-add No Support is emulated in software No Correctly-rounded divide and sqrt operations No Double-precision Floating-point support (cl_khr_fp64) Denormals Yes Infinity and NANs Yes Round to nearest Yes Round to zero Yes Round to infinity Yes IEEE754-2008 fused multiply-add Yes Support is emulated in software No Address bits 64, Little-Endian Global memory size 15619145728 (14.55GiB) Error Correction support No Max memory allocation 3904786432 (3.637GiB) Unified memory for Host and Device No Minimum alignment for any data type 128 bytes Alignment of base address 32768 bits (4096 bytes) Global Memory cache type None Image support No Local memory type Local Local memory size 65536 (64KiB) Max number of constant args 16 Max constant buffer size 67108864 (64MiB) Max size of kernel argument 1024 Queue properties Out-of-order execution No Profiling Yes Profiling timer resolution 0ns Execution capabilities Run OpenCL kernels Yes Run native kernels No Device Extensions cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp64 cl_khr_extended_versioning NULL platform behavior clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) Clover clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [MESA] clCreateContext(NULL, ...) [default] Success [MESA] clCreateContext(NULL, ...) [other] Success [MESA] clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1) Platform Name Clover Device Name RENOIR (renoir, LLVM 15.0.7, DRM 3.54, 6.5.3-060503-generic) clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1) Platform Name Clover Device Name RENOIR (renoir, LLVM 15.0.7, DRM 3.54, 6.5.3-060503-generic) clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1) Platform Name Clover Device Name RENOIR (renoir, LLVM 15.0.7, DRM 3.54, 6.5.3-060503-generic) ICD loader properties ICD loader Name OpenCL ICD Loader ICD loader Vendor OCL Icd free software ICD loader Version 2.2.11 ICD loader Profile OpenCL 2.1 ```

Even this very simple OpenCV program halts the Renoir GPU:

// cl_error.cpp
#include <opencv2/opencv.hpp>
int main(int argc, char** argv)
{
  cv::UMat m(8, 8, CV_8UC3, cv::Scalar::all(0));
  return 0;
}

(compile with g++ cl_error.cpp -lopencv_core, library from libopencv-core-dev).

What the program does is allocate GPU memory and then run a simple kernel to assign zeros to this memory:

__kernel void set(__global uchar* dstptr, int dststep, int dstoffset,
                  int rows, int cols, dstST value_)
{
    int x = get_global_id(0);
    int y0 = get_global_id(1) * rowsPerWI;

    if (x < cols)
    {
        int dst_index  = mad24(y0, dststep, mad24(x, (int)sizeof(dstT1) * cn, dstoffset));

        for (int y = y0, y1 = min(rows, y0 + rowsPerWI); y < y1; ++y, dst_index += dststep)
            storedst(value);
    }
}

My system runs AMD Ryzen 7 PRO 4750U with only integrated GPU (Vega 7), Ubuntu 18.04, kernel 6.5.3. Mesa 22.3.6 and OpenCL via mesa-opencl-icd.

Dmesg output after the program compilation times out is ``` [364724.113497] [drm] scheduler comp_1.3.1 is not ready, skipping [364784.311315] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring comp_1.1.0 timeout, signaled seq=76, emitted seq=77 [364784.312199] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process pid 0 thread pid 0 [364784.313042] amdgpu 0000:06:00.0: amdgpu: GPU reset begin! [364784.413555] amdgpu 0000:06:00.0: amdgpu: MODE2 reset [364784.413620] amdgpu 0000:06:00.0: amdgpu: GPU reset succeeded, trying to resume [364784.413797] [drm] PCIE GART of 1024M enabled. [364784.413801] [drm] PTB located at 0x000000F47FC00000 [364784.413904] [drm] PSP is resuming... [364785.116315] [drm] reserve 0x400000 from 0xf47f800000 for PSP TMR [364785.402598] amdgpu 0000:06:00.0: amdgpu: RAS: optional ras ta ucode is not available [364785.413746] amdgpu 0000:06:00.0: amdgpu: RAP: optional rap ta ucode is not available [364785.413756] amdgpu 0000:06:00.0: amdgpu: SECUREDISPLAY: securedisplay ta ucode is not available [364785.413765] amdgpu 0000:06:00.0: amdgpu: SMU is resuming... [364785.414876] amdgpu 0000:06:00.0: amdgpu: SMU is resumed successfully! [364785.415336] [drm] DMUB hardware initialized: version=0x01010026 [364785.635218] [drm] kiq ring mec 2 pipe 1 q 0 [364785.642132] [drm] VCN decode and encode initialized successfully(under DPG Mode). [364785.642175] [drm] JPEG decode initialized successfully. [364785.642179] amdgpu 0000:06:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0 [364785.642182] amdgpu 0000:06:00.0: amdgpu: ring gfx_low uses VM inv eng 1 on hub 0 [364785.642184] amdgpu 0000:06:00.0: amdgpu: ring gfx_high uses VM inv eng 4 on hub 0 [364785.642187] amdgpu 0000:06:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 5 on hub 0 [364785.642189] amdgpu 0000:06:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 6 on hub 0 [364785.642190] amdgpu 0000:06:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 7 on hub 0 [364785.642192] amdgpu 0000:06:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 8 on hub 0 [364785.642194] amdgpu 0000:06:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 9 on hub 0 [364785.642196] amdgpu 0000:06:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 10 on hub 0 [364785.642198] amdgpu 0000:06:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 11 on hub 0 [364785.642200] amdgpu 0000:06:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 12 on hub 0 [364785.642202] amdgpu 0000:06:00.0: amdgpu: ring kiq_0.2.1.0 uses VM inv eng 13 on hub 0 [364785.642204] amdgpu 0000:06:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 8 [364785.642206] amdgpu 0000:06:00.0: amdgpu: ring vcn_dec uses VM inv eng 1 on hub 8 [364785.642208] amdgpu 0000:06:00.0: amdgpu: ring vcn_enc0 uses VM inv eng 4 on hub 8 [364785.642210] amdgpu 0000:06:00.0: amdgpu: ring vcn_enc1 uses VM inv eng 5 on hub 8 [364785.642212] amdgpu 0000:06:00.0: amdgpu: ring jpeg_dec uses VM inv eng 6 on hub 8 [364786.647420] amdgpu 0000:06:00.0: [drm:amdgpu_ib_ring_tests [amdgpu]] *ERROR* IB test failed on comp_1.2.0 (-110). [364786.651825] amdgpu 0000:06:00.0: amdgpu: ib ring test failed (-110). [364786.761134] amdgpu 0000:06:00.0: amdgpu: MODE2 reset [364786.761210] amdgpu 0000:06:00.0: amdgpu: GPU reset succeeded, trying to resume [364786.761430] [drm] PCIE GART of 1024M enabled. [364786.761436] [drm] PTB located at 0x000000F47FC00000 [364786.761470] [drm] PSP is resuming... [364787.466133] [drm] reserve 0x400000 from 0xf47f800000 for PSP TMR [364787.758019] amdgpu 0000:06:00.0: amdgpu: RAS: optional ras ta ucode is not available [364787.769326] amdgpu 0000:06:00.0: amdgpu: RAP: optional rap ta ucode is not available [364787.769334] amdgpu 0000:06:00.0: amdgpu: SECUREDISPLAY: securedisplay ta ucode is not available [364787.769342] amdgpu 0000:06:00.0: amdgpu: SMU is resuming... [364787.770200] amdgpu 0000:06:00.0: amdgpu: SMU is resumed successfully! [364787.770759] [drm] DMUB hardware initialized: version=0x01010026 [364787.975944] [drm] kiq ring mec 2 pipe 1 q 0 [364787.983608] [drm] VCN decode and encode initialized successfully(under DPG Mode). [364787.983712] [drm] JPEG decode initialized successfully. [364787.983716] amdgpu 0000:06:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0 [364787.983719] amdgpu 0000:06:00.0: amdgpu: ring gfx_low uses VM inv eng 1 on hub 0 [364787.983721] amdgpu 0000:06:00.0: amdgpu: ring gfx_high uses VM inv eng 4 on hub 0 [364787.983723] amdgpu 0000:06:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 5 on hub 0 [364787.983725] amdgpu 0000:06:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 6 on hub 0 [364787.983727] amdgpu 0000:06:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 7 on hub 0 [364787.983729] amdgpu 0000:06:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 8 on hub 0 [364787.983731] amdgpu 0000:06:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 9 on hub 0 [364787.983733] amdgpu 0000:06:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 10 on hub 0 [364787.983735] amdgpu 0000:06:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 11 on hub 0 [364787.983737] amdgpu 0000:06:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 12 on hub 0 [364787.983739] amdgpu 0000:06:00.0: amdgpu: ring kiq_0.2.1.0 uses VM inv eng 13 on hub 0 [364787.983741] amdgpu 0000:06:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 8 [364787.983743] amdgpu 0000:06:00.0: amdgpu: ring vcn_dec uses VM inv eng 1 on hub 8 [364787.983745] amdgpu 0000:06:00.0: amdgpu: ring vcn_enc0 uses VM inv eng 4 on hub 8 [364787.983747] amdgpu 0000:06:00.0: amdgpu: ring vcn_enc1 uses VM inv eng 5 on hub 8 [364787.983748] amdgpu 0000:06:00.0: amdgpu: ring jpeg_dec uses VM inv eng 6 on hub 8 [364787.986529] amdgpu 0000:06:00.0: amdgpu: recover vram bo from shadow start [364787.986531] amdgpu 0000:06:00.0: amdgpu: recover vram bo from shadow done [364787.986550] amdgpu 0000:06:00.0: amdgpu: GPU reset(20) succeeded! ```

So I think these newer architectures might actually need something different than the older tahiti compiler. The question is, obviously, what is the missing part.

lorn10 commented 9 months ago

By the way we have now finally an official merge or pull request regarding the missing support of newer AMD GPUs in libclc. Check #78884 for more details.

And it really looks that it will cover even more of those missing AMD GPU symlinks. :smiley:

However, when someone (like @peci1) will test this with newer Mesa and newer LLVM then be aware of the following fact. The new LLVM pass manager which is required since LLVM 17 has some drawbacks with clover. This was reported in the corresponding Mesa MR 24879 clover/llvm: move to modern pass manager.

So until LLVM 16 that Mesa MR 24879 change can be theoretically reverted and clover will then use again the legacy pass manager. But for LLVM 17 and newer this is no longer possible. Note, - the new LLVM pass manager support landed in Mesa 23.3.0 so the previous Mesa 23.2 is the last release which supported the older one.

peci1 commented 9 months ago

@lorn10 Hmm, I wanted to test, but apparently I don't know enough to do it.

I'm running Ubuntu 20.04 with Mesa 23.3.3 from https://launchpad.net/~kisak/+archive/ubuntu/kisak-mesa/+packages . This normally works, except OpenCL.

So I downloaded the sources for LLVM (current git), SPIRV-LLVM-translator (current git) and mesa 23.3.3 (pristine tar from the above PPA, debian patches not applied).

I've built and installed these into a custom folder (building llvm/libclc, too). But it seems I'm still missing something and a mixture of new and old OpenCL is achieved when I run the example from https://github.com/llvm/llvm-project/issues/52665#issuecomment-1732658768 with LD_LIBRARY_PATH and PATH set to the custom install dir.

When I run the example, I get:

: CommandLine Error: Option 'openmp-ir-builder-optimistic-attributes' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options

I tried to comment out the fatal errors for multiply defined attributes in llvm source code. This lets clover continue a bit longer, but segfaults later in pipe_radeonsi PassManager code:

Thread 1 "a.out" received signal SIGSEGV, Segmentation fault.
llvm::PMTopLevelManager::addImmutablePass (this=0x555555615a70, P=0x555555616100)
    at /usr/include/c++/9/bits/stl_iterator.h:803
803       __normal_iterator(const _Iterator& __i) _GLIBCXX_NOEXCEPT
(gdb) bt
#0  llvm::PMTopLevelManager::addImmutablePass (this=0x555555615a70, P=0x555555616100) at /usr/include/c++/9/bits/stl_iterator.h:803
#1  0x00007ffff305dead in llvm::PMTopLevelManager::schedulePass (this=0x555555615a70, P=<optimized out>) at /tmp/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:745
#2  0x00007fffe0bff2ec in ac_create_passmgr () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#3  0x00007fffe0c0375d in ac_init_llvm_compiler () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#4  0x00007fffe0a59f95 in si_init_compiler () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#5  0x00007fffe0a5d955 in radeonsi_screen_create_impl () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#6  0x00007fffe0b7e4ab in amdgpu_winsys_create () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#7  0x00007fffe0a5e71d in radeonsi_screen_create () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#8  0x00007fffe083b510 in pipe_radeonsi_create_screen () from /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
#9  0x00007ffff6f04185 in pipe_loader_drm_create_screen () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#10 0x00007ffff6f03634 in pipe_loader_create_screen_vk () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#11 0x00007ffff6f0366b in pipe_loader_create_screen () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#12 0x00007ffff6ec49d9 in clover::device::device(clover::platform&, pipe_loader_device*) () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#13 0x00007ffff6ef2993 in clover::intrusive_ref<clover::device> clover::create<clover::device, clover::platform&, pipe_loader_device*&>(clover::platform&, pipe_loader_device*&) ()
   from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#14 0x00007ffff6ef20c5 in clover::platform::platform() () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#15 0x00007ffff6e6d6a8 in __static_initialization_and_destruction_0(int, int) () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#16 0x00007ffff6e6da34 in _GLOBAL__sub_I_platform.cpp () from /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
#17 0x00007ffff7fe0b9a in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7fffffffd6c8, env=env@entry=0x7fffffffd6d8) at dl-init.c:72
#18 0x00007ffff7fe0ca1 in call_init (env=0x7fffffffd6d8, argv=0x7fffffffd6c8, argc=1, l=<optimized out>) at dl-init.c:30
#19 _dl_init (main_map=0x555555579980, argc=1, argv=0x7fffffffd6c8, env=0x7fffffffd6d8) at dl-init.c:119
#20 0x00007ffff756eba5 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:182
#21 0x00007ffff7fe50cf in dl_open_worker (a=a@entry=0x7fffffffcd20) at dl-open.c:758
#22 0x00007ffff756eb48 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:208
#23 0x00007ffff7fe460a in _dl_open (file=0x55555557a940 "libMesaOpenCL.so.1", mode=-2147483647, caller_dlopen=<optimized out>, nsid=-2, argc=1, argv=0x7fffffffd6c8, env=0x7fffffffd6d8) at dl-open.c:837
#24 0x00007ffff7f4834c in dlopen_doit (a=a@entry=0x7fffffffcf40) at dlopen.c:66
#25 0x00007ffff756eb48 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffcee0, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:208
#26 0x00007ffff756ec13 in __GI__dl_catch_error (objname=0x555555570f30, errstring=0x555555570f38, mallocedp=0x555555570f28, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:227
#27 0x00007ffff7f48b59 in _dlerror_run (operate=operate@entry=0x7ffff7f482f0 <dlopen_doit>, args=args@entry=0x7fffffffcf40) at dlerror.c:170
#28 0x00007ffff7f483da in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#29 0x00007ffff720423f in ?? () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#30 0x00007ffff720438d in ?? () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#31 0x00007ffff72048da in ?? () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#32 0x00007ffff72050d3 in clGetPlatformIDs () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#33 0x00007ffff7bf2675 in cv::ocl::haveOpenCL () at ../modules/core/src/ocl.cpp:895
#34 0x00007ffff7bfa968 in cv::ocl::useOpenCL () at ../modules/core/src/ocl.cpp:917
#35 0x00007ffff7c5caad in cv::UMat::getStdAllocator () at ../modules/core/src/umatrix.cpp:234
#36 0x00007ffff7c5d06d in cv::UMat::create (this=0x7fffffffd560, d=<optimized out>, _sizes=<optimized out>, _type=16, _usageFlags=<optimized out>) at ../modules/core/src/umatrix.cpp:441
#37 0x000055555555782b in cv::UMat::create(int, int, int, cv::UMatUsageFlags) ()
#38 0x0000555555557766 in cv::UMat::UMat(int, int, int, cv::Scalar_<double> const&, cv::UMatUsageFlags) ()
#39 0x00005555555573d3 in main ()

Any ideas what else is missing to get a working example?

I use the following config for llvm: cmake -S llvm -B build -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_PROJECTS="clang;libclc" -DCMAKE_INSTALL_PREFIX=/usr/local/clang-17 -DLLVM_ENABLE_RTTI=ON -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_USE_LINKER=lld -DLLVM_PARALLEL_LINK_JOBS=6

And this one for mesa: PKG_CONFIG_PATH=/usr/local/clang-17/lib/pkgconfig PATH=/usr/local/clang-17/bin:$PATH LC_ALL=C.UTF-8 meson .. --buildtype=plain --prefix=/usr/local/clang-17 --sysconfdir=/usr/local/clang-17/etc --localstatedir=/var --libdir=lib/x86_64-linux-gnu -Ddri-drivers-path=/usr/local/clang-17/usr/lib/x86_64-linux-gnu/dri -Ddri-search-path=/usr/local/clang-17/usr/lib/x86_64-linux-gnu/dri:/usr/lib/dri -Dvulkan-drivers="['amd','virtio','amd','swrast']" -Dvulkan-layers="['device-select','intel-nullhw','overlay']" -Dglvnd=true -Dshared-glapi=enabled -Dgallium-omx=disabled -Db_ndebug=false -Dbuild-tests=false -Dtools=drm-shim -Dglx-direct=true -Dgbm=enabled -Ddri3=enabled "-Dplatforms=x11" -Dgallium-extra-hud=true -Dgallium-vdpau=enabled -Dlmsensors=enabled -Dgallium-xa=disabled -Dintel-clc=disabled -Dllvm=enabled -Dgallium-opencl=icd -Dgallium-nine=true -Dgallium-va=enabled "-Dvideo-codecs=vc1dec, h264dec, h264enc, h265dec, h265enc" -Dgallium-drivers="['swrast','r300','r600','radeonsi','zink']" -Dgles1=disabled -Dgles2=enabled -Dosmesa=true -Dsse2=true -Dvalgrind=enabled -Dshared-llvm=true -Dcpp_rtti=true

arsenm commented 9 months ago

Any ideas what else is missing to get a working example?

These are the symptoms of mismatched or multiple versions of llvm linked in the same process

peci1 commented 9 months ago

That's also what I thought. Yet:

(gdb) info sharedlibrary 
From                To                  Syms Read   Shared Object Library
0x00007ffff7fd0100  0x00007ffff7ff2684  Yes         /lib64/ld-linux-x86-64.so.2
0x00007ffff7a4baa0  0x00007ffff7ce8930  Yes         /lib/x86_64-linux-gnu/libopencv_core.so.4.2
0x00007ffff76a4540  0x00007ffff77cd3a2  Yes (*)     /lib/x86_64-linux-gnu/libstdc++.so.6
0x00007ffff7f506c0  0x00007ffff7f6b1b5  Yes (*)     /lib/x86_64-linux-gnu/libgcc_s.so.1
0x00007ffff7430630  0x00007ffff75a54bd  Yes         /lib/x86_64-linux-gnu/libc.so.6
0x00007ffff7f48220  0x00007ffff7f49179  Yes         /lib/x86_64-linux-gnu/libdl.so.2
0x00007ffff7f28ae0  0x00007ffff7f38535  Yes         /lib/x86_64-linux-gnu/libpthread.so.0
0x00007ffff7f08280  0x00007ffff7f18f9b  Yes (*)     /lib/x86_64-linux-gnu/libz.so.1
0x00007ffff7ecfbb0  0x00007ffff7ef0a0c  Yes (*)     /lib/x86_64-linux-gnu/libtbb.so.2
0x00007ffff78be3c0  0x00007ffff7964fa8  Yes         /lib/x86_64-linux-gnu/libm.so.6
0x00007ffff7204180  0x00007ffff720671e  Yes (*)     /lib/x86_64-linux-gnu/libOpenCL.so.1
0x00007ffff6e2daf0  0x00007ffff706a893  Yes (*)     /usr/local/clang-17/lib/x86_64-linux-gnu/libMesaOpenCL.so.1
0x00007ffff7e16240  0x00007ffff7ea7d0a  Yes (*)     /lib/x86_64-linux-gnu/libzstd.so.1
0x00007ffff7df9580  0x00007ffff7e0a115  Yes (*)     /lib/x86_64-linux-gnu/libelf.so.1
0x00007fffefcf5bc0  0x00007ffff6848c83  Yes         /usr/local/clang-17/lib/libclang-cpp.so.18git
0x00007ffff7de5740  0x00007ffff7def739  Yes (*)     /lib/x86_64-linux-gnu/libdrm.so.2
0x00007ffff7db6230  0x00007ffff7dd1b47  Yes (*)     /lib/x86_64-linux-gnu/libexpat.so.1
0x00007fffe71c1740  0x00007fffeb512033  Yes         /usr/local/clang-17/lib/libLLVM-18git.so
0x00007ffff78a9720  0x00007ffff78acd70  Yes         /lib/x86_64-linux-gnu/librt.so.1
0x00007fffe3873e50  0x00007fffe399cb8e  Yes (*)     /lib/x86_64-linux-gnu/libxml2.so.2
0x00007fffe36c5920  0x00007fffe37a9967  Yes (*)     /lib/x86_64-linux-gnu/libicuuc.so.66
0x00007ffff78813c0  0x00007ffff78983b6  Yes (*)     /lib/x86_64-linux-gnu/liblzma.so.5
0x00007fffe1a01040  0x00007fffe1a010f9  Yes (*)     /lib/x86_64-linux-gnu/libicudata.so.66
0x00007fffe083b3d0  0x00007fffe119f049  Yes (*)     /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_radeonsi.so
0x00007ffff78712a0  0x00007ffff7877b1d  Yes (*)     /lib/x86_64-linux-gnu/libdrm_radeon.so.1
0x00007ffff71f7760  0x00007ffff71fbf11  Yes (*)     /lib/x86_64-linux-gnu/libdrm_amdgpu.so.1
0x00007fffdfe2c5b0  0x00007fffe02a9099  Yes (*)     /usr/local/clang-17/lib/x86_64-linux-gnu/gallium-pipe/pipe_swrast.so

Do you see anything pulling in old llvm from the system?

peci1 commented 9 months ago

Ahh, libdrm? Is it possible to use a different version than the one from running system? I don't want to break my base system by installing to /usr...

peci1 commented 9 months ago

The result is the same even with libdrm 2.4.0 compiled from source.

Or, do I have to build mesa and libdrm with the custom-built llvm? Or can the system gcc be used for that?

arsenm commented 9 months ago

The result is the same even with libdrm 2.4.0 compiled from source.

Or, do I have to build mesa and libdrm with the custom-built llvm? Or can the system gcc be used for that?

The host compiler doesn't matter. You should be building mesa/libdrm against the custom built

peci1 commented 9 months ago

The host compiler doesn't matter. You should be building mesa/libdrm against the custom built

Mesa says:

 LLVM
    Enabled                      : YES
    Version                      : 18.0.0

libdrm doesn't seem to even depend on llvm (at least not in configure phase)

lorn10 commented 9 months ago

Thanks @peci1 for the testing. :+1:

As far I know we have on older Debian/Ubuntu some strange problems with newer (and especially mixed) LLVM versions. For example, even 22.04 LTS is in the oibaf PPA limited to LLVM 15. According to some comments on Mesa the underlying problem is purely Debian related and not reproducible on other distros.

Therefore I think if 22.04 LTS has those LLVM flaws, 20.04 LTS might be even more critical. :thinking:

So instead of installing the newest LLVM the libclc patch can be used within a LLVM 15 installation. I know that there exist a way to add individual patches to LLVM when it is build.

Whatever, I am on 22.04 and I simply wait until the libclc patch will be included in the oibaf PPA Jammy branch. And yes, this would be possible also regarding the kisak PPA.

Note, - Mesa devel 24.1 needs at least LLVM 15 when OpenCL support is included. This was changed recently because the older LLVM support was already broken.

peci1 commented 9 months ago

Hmm, I tried building LLVM 15 from the PPA with the AMD patch, but still no success. Still getting the multiply registered options.

$ PATH=/usr/local/clang-15/bin:$PATH LD_LIBRARY_PATH=/usr/local/clang-15/lib:/usr/local/clang-15/lib/x86_64-linux-gnu:/usr/local/clang-15/usr/lib:/usr/local/clang-15/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH gdb clinfo
GNU gdb (Ubuntu 10.2-0ubuntu1~20.04~1) 10.2
(gdb) start
Function "main" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Temporary breakpoint 1 (-qualified main) pending.
Starting program: /usr/bin/clinfo 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
: CommandLine Error: Option 'use-dbg-addr' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: Adresář nebo soubor neexistuje.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7a30859 in __GI_abort () at abort.c:79
#2  0x00007fffe7a63e6e in llvm::report_fatal_error(llvm::Twine const&, bool) [clone .cold] () from /usr/local/clang-15/lib/libLLVM-15.so.1
#3  0x00007fffe7a6407e in llvm::report_fatal_error(char const*, bool) () from /usr/local/clang-15/lib/libLLVM-15.so.1
#4  0x00007fffe7a3e0c6 in (anonymous namespace)::CommandLineParser::addOption(llvm::cl::Option*, llvm::cl::SubCommand*) () from /usr/local/clang-15/lib/libLLVM-15.so.1
#5  0x00007fffe7a3e20f in llvm::cl::Option::addArgument() () from /usr/local/clang-15/lib/libLLVM-15.so.1
#6  0x00007fffe7c316c4 in _GLOBAL__sub_I_DIBuilder.cpp () from /usr/local/clang-15/lib/libLLVM-15.so.1
#7  0x00007ffff7fe0b9a in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7fffffffd478, env=env@entry=0x7fffffffd488) at dl-init.c:72
#8  0x00007ffff7fe0ca1 in call_init (env=0x7fffffffd488, argv=0x7fffffffd478, argc=1, l=<optimized out>) at dl-init.c:30
#9  _dl_init (main_map=0x555555578550, argc=1, argv=0x7fffffffd478, env=0x7fffffffd488) at dl-init.c:119
#10 0x00007ffff7b6eba5 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:182
#11 0x00007ffff7fe50cf in dl_open_worker (a=a@entry=0x7fffffffcec0) at dl-open.c:758
#12 0x00007ffff7b6eb48 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:208
#13 0x00007ffff7fe460a in _dl_open (file=0x555555579520 "libMesaOpenCL.so.1", mode=-2147483647, caller_dlopen=<optimized out>, nsid=-2, argc=1, argv=0x7fffffffd478, env=0x7fffffffd488) at dl-open.c:837
#14 0x00007ffff7f6d34c in dlopen_doit (a=a@entry=0x7fffffffd0e0) at dlopen.c:66
#15 0x00007ffff7b6eb48 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffd080, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:208
#16 0x00007ffff7b6ec13 in __GI__dl_catch_error (objname=0x7ffff7f710f0 <last_result+16>, errstring=0x7ffff7f710f8 <last_result+24>, mallocedp=0x7ffff7f710e8 <last_result+8>, operate=<optimized out>, 
    args=<optimized out>) at dl-error-skeleton.c:227
#17 0x00007ffff7f6db59 in _dlerror_run (operate=operate@entry=0x7ffff7f6d2f0 <dlopen_doit>, args=args@entry=0x7fffffffd0e0) at dlerror.c:170
#18 0x00007ffff7f6d3da in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#19 0x00007ffff7c0423f in ?? () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#20 0x00007ffff7c0438d in ?? () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#21 0x00007ffff7c048da in ?? () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#22 0x00007ffff7c050d3 in clGetPlatformIDs () from /lib/x86_64-linux-gnu/libOpenCL.so.1
#23 0x000055555555b478 in ?? ()
#24 0x00007ffff7a32083 in __libc_start_main (main=0x55555555b2c0, argc=1, argv=0x7fffffffd478, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd468)
    at ../csu/libc-start.c:308
#25 0x000055555555b7ba in ?? ()
(gdb) info sharedlibrary 
From                To                  Syms Read   Shared Object Library
0x00007ffff7fd0100  0x00007ffff7ff2684  Yes         /lib64/ld-linux-x86-64.so.2
0x00007ffff7c04180  0x00007ffff7c0671e  Yes (*)     /lib/x86_64-linux-gnu/libOpenCL.so.1
0x00007ffff7f6d220  0x00007ffff7f6e179  Yes         /lib/x86_64-linux-gnu/libdl.so.2
0x00007ffff7a30630  0x00007ffff7ba54bd  Yes         /lib/x86_64-linux-gnu/libc.so.6
0x00007ffff705ac70  0x00007ffff73fdf32  Yes (*)     /lib/x86_64-linux-gnu/libMesaOpenCL.so.1
0x00007ffff7f4f280  0x00007ffff7f5ff9b  Yes (*)     /lib/x86_64-linux-gnu/libz.so.1
0x00007ffff7ea8240  0x00007ffff7f39d0a  Yes (*)     /lib/x86_64-linux-gnu/libzstd.so.1
0x00007ffff7e8b580  0x00007ffff7e9c115  Yes (*)     /lib/x86_64-linux-gnu/libelf.so.1
0x00007ffff002d520  0x00007ffff654a7a3  Yes (*)     /usr/local/clang-15/lib/libclang-cpp.so.15
0x00007ffff7e77740  0x00007ffff7e81739  Yes (*)     /lib/x86_64-linux-gnu/libdrm.so.2
0x00007ffff7e48230  0x00007ffff7e63b47  Yes (*)     /lib/x86_64-linux-gnu/libexpat.so.1
0x00007fffe79c0800  0x00007fffeb7ddd93  Yes (*)     /usr/local/clang-15/lib/libLLVM-15.so.1
0x00007fffe24e3970  0x00007fffe3e36784  Yes (*)     /usr/local/clang-15/lib/libLLVMSPIRVLib.so.15
0x00007fffe18a4540  0x00007fffe19cd3a2  Yes (*)     /lib/x86_64-linux-gnu/libstdc++.so.6
0x00007ffff78cc3c0  0x00007ffff7972fa8  Yes         /lib/x86_64-linux-gnu/libm.so.6
0x00007ffff7e226c0  0x00007ffff7e3d1b5  Yes (*)     /lib/x86_64-linux-gnu/libgcc_s.so.1
0x00007ffff78a2ae0  0x00007ffff78b2535  Yes         /lib/x86_64-linux-gnu/libpthread.so.0
0x00007ffff7e17720  0x00007ffff7e1ad70  Yes         /lib/x86_64-linux-gnu/librt.so.1
0x00007fffe1673e50  0x00007fffe179cb8e  Yes (*)     /lib/x86_64-linux-gnu/libxml2.so.2
0x00007fffe14c5920  0x00007fffe15a9967  Yes (*)     /lib/x86_64-linux-gnu/libicuuc.so.66
0x00007ffff78763c0  0x00007ffff788d3b6  Yes (*)     /lib/x86_64-linux-gnu/liblzma.so.5
0x00007fffdf801040  0x00007fffdf8010f9  Yes (*)     /lib/x86_64-linux-gnu/libicudata.so.66

I've noticed the addOption() function is called by the static initialization of three libraries: libLLVM-15.so.1, libLLVMSPIRVLib.so.15 and libclang-cpp.so.15. Not sure if this is correct.

lorn10 commented 9 months ago

Yeah I have not really a clue about that LLVM stuff. However, I would try to revert anything (LLVM, kisak PPA) to stock and then do an upgrade to 22.04 LTS. This would open the door to use the oibaf PPA.

Perhaps @FabioPedretti has any idea if the libclc patch can be integrated into the LLVM 15 Jammy branch. And most likely an upgrade to 22.04 LTS might generally improve the situation with newer LLVM versions even if Jammy is in the meantime also quite old.

FabioPedretti commented 9 months ago

I don't plan to try to patch llvm 15. But if a patched version of llvm 15 appears in some Ubuntu version (for example on 24.04/noble), I could try to backport it to 22.04/jammy.

lorn10 commented 9 months ago

Probably it would really make sense to include that libclc update also into any further (not planned) LLVM 17.0.x or LLVM 16.0.x release.

So for the case that there exist somewhere an "informal" LLVM backport fix list, - please include also MR https://github.com/llvm/llvm-project/pull/78884. @arsenm :+1:

arsenm commented 9 months ago

Can this be closed? There won't be any more 15/16/17 releases

lorn10 commented 9 months ago

The underlying issue is resolved at least for the upcoming LLVM 18 so yes this can be closed.

And for older LLVM versions there exist the possibility to re-build them with that #78884 patch.

(As long as the underlying distro isn't too old.)