RDNA2+ cards seem to be underutilized, because of dual compute unit design?

[pigmej]

From my understanding of RDNA2 architecture each RDNA2 (and newer) will report only half of CUs to the OpenCL.

(from the whitepaper)

The new dual compute unit design is the essence of the RDNA architecture and replaces the GCN compute unit as the fundamental computational building block of the GPU. As Figure 2 illustrates, the dual compute unit still comprises four SIMDs that operate independently. However, this dual compute unit was specifically designed for wave32 mode; the RDNA SIMDs include 32 ALUs, twice as wide as the vector ALUS in the prior generation, boosting performance by executing a wavefront up to 2X faster. The new SIMDs were built for mixedprecision operation and efficiently compute with a variety of datatypes to enable scientific computing and machine learning. Figure 3 below illustrates how the new dual compute unit exploits instruction-level parallelism within a simple example shader to execute on a SIMD with half the latency in wave32 mode and a 44% reduction in latency for wave64 mode compared to the previous generation GCN SIMD.

The result is that it seems that the GPUs are underutilized.

[pigmej]

7900xtx reports:

2024-02-13T13:37:42.980+0100    INFO    selecting 0 provider from 2 available   {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 321}
2024-02-13T13:37:42.980+0100    INFO    Using provider: [GPU] AMD Accelerated Parallel Processing/gfx1100       {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 334}
2024-02-13T13:37:42.980+0100    INFO    device memory: 24560 MB, max_mem_alloc_size: 20876 MB, max_compute_units: 48, max_wg_size: 256       {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 152}
2024-02-13T13:37:43.763+0100    INFO    preferred_wg_size_multiple: 32, kernel_wg_size: 256     {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 186}
2024-02-13T13:37:43.763+0100    INFO    Using: global_work_size: 41728, local_work_size: 32     {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 199}
2024-02-13T13:37:43.763+0100    INFO    Allocating buffer for input: 32 bytes   {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 203}
2024-02-13T13:37:43.763+0100    INFO    Allocating buffer for output: 1335296 bytes     {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 211}
2024-02-13T13:37:43.763+0100    INFO    Allocating buffer for lookup: 21877489664 bytes {"module": "scrypt_ocl", "file": "scrypt-ocl\\src\\lib.rs", "line": 219}

so it reports 48 CU while it have 96 actually.

Please note that the preferred_wg_size_multiple: 32 also suggests that the actual num is 96.

[pigmej]

Please note that the 48CU is only used here as log message. The important part is max_mem_alloc_size and global|local_work_size

[pigmej]

On 7900xtx you can get more performance by setting LOOKUP_GAP to 4 and that yields about 25% perf improvement.

I think for RDNA2 cards where we have high ram to CU ratio we're packing too much ram. In comparsion nvidia 4090 allows us to use 6GiB of ram (max_mem_alloc_size) while 7900xtx 21GiB.