[Issue]: OpenCL fails on a Ryzen 5700G APU on ROCm 6.0.0 - regression (OpenCL buffer operations cause a ring sdma0 timeout & GPU reset)

[joanbm]

Problem Description

After upgrading my Arch Linux system with a Ryzen 5700G APU to ROCm 6.0.0, running pretty much any OpenCL application (see below for an example) fails.

In particular, creating an OpenCL command queue, building a kernel, etc. works fine, but when trying to run operations over OpenCL buffers (i.e. clEnqueueReadBuffer, clEnqueueWriteBuffer), the application in the best scenario hangs (when running it on a TTY), and in the worst scenario causes a GPU reset (when running it on a graphical session). An error message *ERROR* ring sdma0 timeout is also logged to dmesg along with more logs related to the GPU reset (full logs included below).

If I downgrade ROCm to 5.7.1, OpenCL applications run fine again.

Another workaround I found is setting the environment variable HSA_ENABLE_SDMA=0. This appears to avoid the problematic code path and OpenCL applications are able to run again.

Bisection

I have tried to bisect the issue and it appears to be related to this new condition introduced between ROCM 5.7.1 and 6.0.0 in src/core/runtime/amd_blit_sdma.cpp:

  // Some GFX9 devices require a minimum of 64 DWORDS per ring buffer submission.
  if (agent_->isa()->GetVersion() >= core::Isa::Version(9, 0, 0) &&
      (agent_->isa()->GetVersion() <= core::Isa::Version(9, 0, 4) ||
       agent_->isa()->GetVersion() == core::Isa::Version(9, 0, 12))) {
    min_submission_size_ = 256;
  }

In particular, this later causes this also newly introduced code to run:

  // Pad size is DWORD aligned since all commands are dword aligned.
  // Insert NOP header DWORD with value of the number of null DWORDs shifted
  // by 16 bits to pad total submission.
  if (pad_size) {
    memset(command_addr, 0, pad_size);
    uint32_t *dword_command_addr = reinterpret_cast<uint32_t*>(command_addr);
    dword_command_addr[total_command_size/4] = (pad_size/4 - 1) << 16;
  }

Commenting the last statement dword_command_addr[total_command_size/4] = (pad_size/4 - 1) << 16; appears to fix the problem as well. I'm not sure if I'm understanding the code correctly, but I think it is potentially an out-of-bounds write? That would also be consistent with it causing a full GPU reset.

Operating System

Arch Linux updated as of 2024-02-08, Linux kernel 6.7.4.

CPU

AMD Ryzen 7 5700G with Radeon Graphics

GPU

Other

Other

AMD Ryzen 7 5700G with Radeon Graphics - i.e. the integrated APU (gfx90c)

ROCm Version

ROCm 6.0.0

ROCm Component

ROCR-Runtime

Steps to Reproduce

Save the following OpenCL application:

#include <iostream>
#include <vector>

#define CL_HPP_ENABLE_EXCEPTIONS
#define CL_HPP_TARGET_OPENCL_VERSION 200
#include <CL/opencl.hpp>

int main() {
    try {
        std::vector<cl::Platform> platforms;
        cl::Platform::get(&platforms);
        if (platforms.empty()) return EXIT_FAILURE;

        std::vector<cl::Device> devices;
        platforms[0].getDevices(CL_DEVICE_TYPE_GPU, &devices);
        if (devices.empty()) return EXIT_FAILURE;

        cl::Device device = devices[0];
        std::cout << "Vendor: " << device.getInfo<CL_DEVICE_VENDOR>() << std::endl;
        std::cout << "Name: " << device.getInfo<CL_DEVICE_NAME>() << std::endl;

        cl::Context context(device);
        cl::CommandQueue queue(context, device);

        std::vector<int> hostData(10, 42);
        cl::Buffer buffer(context, CL_MEM_READ_WRITE | CL_MEM_COPY_HOST_PTR,
                          sizeof(int) * hostData.size(), hostData.data());
        queue.enqueueWriteBuffer(buffer, CL_TRUE, 0, sizeof(int) * hostData.size(), hostData.data());
        std::vector<int> outputData(hostData.size());
        queue.enqueueReadBuffer(buffer, CL_TRUE, 0, sizeof(int) * hostData.size(), outputData.data());

        std::cout << "Buffer data: ";
        for (int i : outputData) {
            std::cout << i << " ";
        }
        std::cout << std::endl;

        std::cout << "Success!" << std::endl;
        return EXIT_SUCCESS;
    } catch (const cl::Error &err) {
        std::cerr
            << "OpenCL error: "
            << err.what() << "(" << err.err() << ")"
            << std::endl;
        return 1;
    }
}

Build it:

$ g++ hello.cpp -o hello -lOpenCL

Run it on a machine with a Ryzen 5700G processor and ROCm 6.0.0:

$ ./hello

Expected Behavior: The program should run correctly and print the following output:

Vendor: Advanced Micro Devices, Inc.
Name: gfx90c:xnack-
Buffer data: 42 42 42 42 42 42 42 42 42 42 
Success!

Actual Behavior: The program prints:

Vendor: Advanced Micro Devices, Inc.
Name: gfx90c:xnack-

Then it hangs, and likely causes a GPU reset.

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
DMAbuf Support:          YES

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD Ryzen 7 5700G with Radeon Graphics
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Ryzen 7 5700G with Radeon Graphics
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4673                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            16                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    31717352(0x1e3f7e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    31717352(0x1e3f7e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    31717352(0x1e3f7e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    gfx90c                             
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Radeon Graphics                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      1024(0x400) KB                     
  Chip ID:                 5688(0x1638)                       
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2000                               
  BDFID:                   12288                              
  Internal Node ID:        1                                  
  Compute Unit:            8                                  
  SIMDs per CU:            4                                  
  Shader Engines:          1                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        40(0x28)                           
  Max Work-item Per CU:    2560(0xa00)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 469                                
  SDMA engine uCode::      40                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    1048576(0x100000) KB               
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    1048576(0x100000) KB               
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90c:xnack-   
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*** Done ***             

Additional Information

dmesg output at the time of the crash:

[  308.334535] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring sdma0 timeout, signaled seq=1255, emitted seq=1257
[  308.334876] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process  pid 0 thread  pid 0
[  308.335146] amdgpu 0000:30:00.0: amdgpu: GPU reset begin!
[  308.335154] amdgpu: Failed to suspend process 0x8010
[  308.545396] amdgpu 0000:30:00.0: amdgpu: MODE2 reset
[  308.545598] amdgpu 0000:30:00.0: amdgpu: GPU reset succeeded, trying to resume
[  308.545743] [drm] PCIE GART of 1024M enabled.
[  308.545746] [drm] PTB located at 0x000000F43FC00000
[  308.545803] [drm] VRAM is lost due to GPU reset!
[  308.545804] [drm] PSP is resuming...
[  309.248396] [drm] reserve 0x400000 from 0xf43f800000 for PSP TMR
[  309.529565] amdgpu 0000:30:00.0: amdgpu: RAS: optional ras ta ucode is not available
[  309.540284] amdgpu 0000:30:00.0: amdgpu: RAP: optional rap ta ucode is not available
[  309.540286] amdgpu 0000:30:00.0: amdgpu: SECUREDISPLAY: securedisplay ta ucode is not available
[  309.540288] amdgpu 0000:30:00.0: amdgpu: SMU is resuming...
[  309.540510] amdgpu 0000:30:00.0: amdgpu: SMU is resumed successfully!
[  309.541037] [drm] DMUB hardware initialized: version=0x01010028
[  309.805376] [drm] kiq ring mec 2 pipe 1 q 0
[  309.807861] [drm] VCN decode and encode initialized successfully(under DPG Mode).
[  309.807903] [drm] JPEG decode initialized successfully.
[  309.807906] amdgpu 0000:30:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0
[  309.807908] amdgpu 0000:30:00.0: amdgpu: ring gfx_low uses VM inv eng 1 on hub 0
[  309.807910] amdgpu 0000:30:00.0: amdgpu: ring gfx_high uses VM inv eng 4 on hub 0
[  309.807911] amdgpu 0000:30:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 5 on hub 0
[  309.807912] amdgpu 0000:30:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 6 on hub 0
[  309.807913] amdgpu 0000:30:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 7 on hub 0
[  309.807914] amdgpu 0000:30:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 8 on hub 0
[  309.807915] amdgpu 0000:30:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 9 on hub 0
[  309.807917] amdgpu 0000:30:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 10 on hub 0
[  309.807918] amdgpu 0000:30:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 11 on hub 0
[  309.807919] amdgpu 0000:30:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 12 on hub 0
[  309.807920] amdgpu 0000:30:00.0: amdgpu: ring kiq_0.2.1.0 uses VM inv eng 13 on hub 0
[  309.807921] amdgpu 0000:30:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 8
[  309.807923] amdgpu 0000:30:00.0: amdgpu: ring vcn_dec uses VM inv eng 1 on hub 8
[  309.807924] amdgpu 0000:30:00.0: amdgpu: ring vcn_enc0 uses VM inv eng 4 on hub 8
[  309.807925] amdgpu 0000:30:00.0: amdgpu: ring vcn_enc1 uses VM inv eng 5 on hub 8
[  309.807926] amdgpu 0000:30:00.0: amdgpu: ring jpeg_dec uses VM inv eng 6 on hub 8
[  309.809563] amdgpu 0000:30:00.0: amdgpu: recover vram bo from shadow start
[  309.809565] amdgpu 0000:30:00.0: amdgpu: recover vram bo from shadow done
[  309.809573] amdgpu 0000:30:00.0: amdgpu: GPU reset(1) succeeded!
[  309.809593] [drm] Skip scheduling IBs!
[  309.809601] [drm] Skip scheduling IBs!
[  309.809605] [drm] Skip scheduling IBs!
[  309.809607] [drm] Skip scheduling IBs!
[  309.809609] [drm] Skip scheduling IBs!
[  309.809613] [drm] Skip scheduling IBs!
[  309.809622] [drm] Skip scheduling IBs!
[  309.809635] [drm] Skip scheduling IBs!
[  309.809643] [drm] Skip scheduling IBs!
[  309.809646] [drm] Skip scheduling IBs!
[  309.809649] [drm] Skip scheduling IBs!
[  309.809652] [drm] Skip scheduling IBs!
[  309.809655] [drm] Skip scheduling IBs!
[  309.809657] [drm] Skip scheduling IBs!
[  309.809660] [drm] Skip scheduling IBs!
[  309.809662] [drm] Skip scheduling IBs!
[  309.809665] [drm] Skip scheduling IBs!
[  309.809668] [drm] Skip scheduling IBs!
[  309.809670] [drm] Skip scheduling IBs!
[  309.809673] [drm] Skip scheduling IBs!
[  309.809675] [drm] Skip scheduling IBs!
[  309.809678] [drm] Skip scheduling IBs!

clinfo output:

Number of platforms                               1
  Platform Name                                   AMD Accelerated Parallel Processing
  Platform Vendor                                 Advanced Micro Devices, Inc.
  Platform Version                                OpenCL 2.1 AMD-APP.dbg (3602.0)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_amd_event_callback 
  Platform Extensions function suffix             AMD
  Platform Host timer resolution                  1ns

  Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 1
  Device Name                                     gfx90c:xnack-
  Device Vendor                                   Advanced Micro Devices, Inc.
  Device Vendor ID                                0x1002
  Device Version                                  OpenCL 2.0 
  Driver Version                                  3602.0 (HSA1.1,LC)
  Device OpenCL C Version                         OpenCL C 2.0 
  Device Type                                     GPU
  Device Board Name (AMD)                         AMD Radeon Graphics
  Device PCI-e ID (AMD)                           0x1638
  Device Topology (AMD)                           PCI-E, 0000:30:00.0
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               8
  SIMD per compute unit (AMD)                     4
  SIMD width (AMD)                                16
  SIMD instruction width (AMD)                    1
  Max clock frequency                             2000MHz
  Graphics IP (AMD)                               9.0
  Device Partition                                (core)
    Max number of sub-devices                     8
    Supported partition types                     None
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             1024x1024x1024
  Max work group size                             256
  Preferred work group size (AMD)                 256
  Max work group size (AMD)                       1024
  Preferred work group size multiple (kernel)     64
  Wavefront width (AMD)                           64
  Preferred / native vector sizes                 
    char                                                 4 / 4       
    short                                                2 / 2       
    int                                                  1 / 1       
    long                                                 1 / 1       
    half                                                 1 / 1        (cl_khr_fp16)
    float                                                1 / 1       
    double                                               1 / 1        (cl_khr_fp64)
  Half-precision Floating-point support           (cl_khr_fp16)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              1073741824 (1024MiB)
  Global free memory (AMD)                        995328 (972MiB) 995328 (972MiB)
  Global memory channels (AMD)                    4
  Global memory banks per channel (AMD)           4
  Global memory bank width (AMD)                  256 bytes
  Error Correction support                        No
  Max memory allocation                           912680544 (870.4MiB)
  Unified memory for Host and Device              No
  Shared Virtual Memory (SVM) capabilities        (core)
    Coarse-grained buffer sharing                 Yes
    Fine-grained buffer sharing                   Yes
    Fine-grained system sharing                   No
    Atomics                                       No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       1024 bits (128 bytes)
  Preferred alignment for atomics                 
    SVM                                           0 bytes
    Global                                        0 bytes
    Local                                         0 bytes
  Max size for global variable                    912680544 (870.4MiB)
  Preferred total size of global vars             1073741824 (1024MiB)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        16384 (16KiB)
  Global Memory cache line size                   64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             16
    Max size for 1D images from buffer            134217728 pixels
    Max 1D or 2D image array size                 8192 images
    Base address alignment for 2D image buffers   256 bytes
    Pitch alignment for 2D image buffers          256 pixels
    Max 2D image size                             16384x16384 pixels
    Max 3D image size                             16384x16384x8192 pixels
    Max number of read image args                 128
    Max number of write image args                8
    Max number of read/write image args           64
  Max number of pipe args                         16
  Max active pipe reservations                    16
  Max pipe packet size                            912680544 (870.4MiB)
  Local memory type                               Local
  Local memory size                               65536 (64KiB)
  Local memory size per CU (AMD)                  65536 (64KiB)
  Local memory banks (AMD)                        32
  Max number of constant args                     8
  Max constant buffer size                        912680544 (870.4MiB)
  Preferred constant buffer size (AMD)            16384 (16KiB)
  Max size of kernel argument                     1024
  Queue properties (on host)                      
    Out-of-order execution                        No
    Profiling                                     Yes
  Queue properties (on device)                    
    Out-of-order execution                        Yes
    Profiling                                     Yes
    Preferred size                                262144 (256KiB)
    Max size                                      8388608 (8MiB)
  Max queues on device                            1
  Max events on device                            1024
  Prefer user sync for interop                    Yes
  Number of P2P devices (AMD)                     0
  Profiling timer resolution                      1ns
  Profiling timer offset since Epoch (AMD)        0ns (Thu Jan  1 00:00:00 1970)
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
    Thread trace supported (AMD)                  No
    Number of async queues (AMD)                  8
    Max real-time compute queues (AMD)            8
    Max real-time compute units (AMD)             8
  printf() buffer size                            4194304 (4MiB)
  Built-in kernels                                (n/a)
  Device Extensions                               cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_image2d_from_buffer cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program 

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  No platform
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   No platform
  clCreateContext(NULL, ...) [default]            No platform
  clCreateContext(NULL, ...) [other]              Success [AMD]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  Success (1)
    Platform Name                                 AMD Accelerated Parallel Processing
    Device Name                                   gfx90c:xnack-
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  Success (1)
    Platform Name                                 AMD Accelerated Parallel Processing
    Device Name                                   gfx90c:xnack-
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  Success (1)
    Platform Name                                 AMD Accelerated Parallel Processing
    Device Name                                   gfx90c:xnack-
    NOTE:   your OpenCL library only supports OpenCL 2.0,
        but some installed platforms support OpenCL 2.1.
        Programs using 2.1 features may crash
        or behave unexpectedly

[shwetagkhatri]

Hi @joanbm , Could you please run your test with the follow patch? This fixes the pad_size address -

+++ b/opensrc/hsa-runtime/core/runtime/amd_blit_sdma.cpp
@@ -471,0 +472 @@ hsa_status_t BlitSdma<RingIndexTy, HwIndexMonotonic, SizeToCountOffset, useGCR>:
 + command_addr += trap_command_size_;
@@ -482 +483 @@ hsa_status_t BlitSdma<RingIndexTy, HwIndexMonotonic, SizeToCountOffset, useGCR>:
- dword_command_addr[total_command_size/4] = (pad_size/4 - 1) << 16;
+ dword_command_addr[0] = (pad_size/4 - 1) << 16;

[joanbm]

@shwetagkhatri Everything seems to work fine with your patch applied (I tried the test case above, hashcat, and some other simple OpenCL programs), thanks.

[shwetagkhatri]

Thanks for confirming @joanbm. This fix will be available with ROCM 6.1 release.