Closed Gardene-el closed 6 months ago
Snektron
commented
6 months ago This is caused by hipcub::DeviceReduce::Sum, which calls into rocprim on AMD. It was previously caused by macro that wasn't properly defined for these architectures, and has been fixed for 6.0 or 6.1. See https://github.com/ROCm/rocPRIM/issues/452. Could you try with an updated rocPRIM installation? The easiest way to do that is to use the rocm/rocm-terminal:6.1 docker image.
Gardene-el
commented
6 months ago I tried to use the rocm/rocm-terminal and avoid the error: Illegal instruction detected, but still in the same project, i got another error, which i guess also a wrapper problem.
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/monte_carlo_pi'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -isystem /opt/rocm/include -isystem /opt/rocm/include -I ../../Common -D__HIP_PLATFORM_AMD__ -L /opt/rocm/lib -o applications_monte_carlo_pi main.hip -lhiprand
In file included from main.hip:25:
../../Common/hiprand_utils.hpp:28:10: fatal error: 'hiprand/hiprand.h' file not found
28 | #include <hiprand/hiprand.h>
| ^~~~~~~~~~~~~~~~~~~
1 error generated when compiling for gfx1034.The whole log:
❯ sudo docker pull rocm/rocm-terminal
sudo docker run -it --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined --group-add video rocm/rocm-terminal
[sudo] password for cr0c0dile:
Using default tag: latest
latest: Pulling from rocm/rocm-terminal
17d0386c2fff: Pull complete
268c8493c252: Pull complete
009ab54bcd2e: Pull complete
944182d77231: Pull complete
4f4fb700ef54: Pull complete
Digest: sha256:803f6250c28d86b3997cd1ef49af238c66e20276139f59d6740075e4915aa0cb
Status: Downloaded newer image for rocm/rocm-terminal:latest
docker.io/rocm/rocm-terminal:latest
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
rocm-user@4edf32dc05ea:~$ git clone https://github.com/ROCm/rocm-examples.git
Cloning into 'rocm-examples'...
remote: Enumerating objects: 8646, done.
remote: Counting objects: 100% (1834/1834), done.
remote: Compressing objects: 100% (546/546), done.
remote: Total 8646 (delta 1496), reused 1476 (delta 1283), pack-reused 6812
Receiving objects: 100% (8646/8646), 1.83 MiB | 2.77 MiB/s, done.
Resolving deltas: 100% (7446/7446), done.
rocm-user@4edf32dc05ea:~$ cd rocm-examples
rocm-user@4edf32dc05ea:~/rocm-examples$ cmake -S . -B build
CMake Error at CMakeLists.txt:23 (cmake_minimum_required):
CMake 3.21.3 or higher is required. You are running version 3.16.3
-- Configuring incomplete, errors occurred!
rocm-user@4edf32dc05ea:~/rocm-examples$ make
make -C Applications
make[1]: Entering directory '/home/rocm-user/rocm-examples/Applications'
make -C bitonic_sort
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/bitonic_sort'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -o applications_bitonic_sort main.hip
make[2]: Leaving directory '/home/rocm-user/rocm-examples/Applications/bitonic_sort'
make -C convolution
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/convolution'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -o applications_convolution main.hip
make[2]: Leaving directory '/home/rocm-user/rocm-examples/Applications/convolution'
make -C floyd_warshall
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/floyd_warshall'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -o applications_floyd_warshall main.hip
make[2]: Leaving directory '/home/rocm-user/rocm-examples/Applications/floyd_warshall'
make -C histogram
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/histogram'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -o applications_histogram main.hip
make[2]: Leaving directory '/home/rocm-user/rocm-examples/Applications/histogram'
make -C prefix_sum
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/prefix_sum'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -o applications_prefix_sum main.hip
make[2]: Leaving directory '/home/rocm-user/rocm-examples/Applications/prefix_sum'
make -C monte_carlo_pi
make[2]: Entering directory '/home/rocm-user/rocm-examples/Applications/monte_carlo_pi'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -isystem /opt/rocm/include -isystem /opt/rocm/include -I ../../Common -D__HIP_PLATFORM_AMD__ -L /opt/rocm/lib -o applications_monte_carlo_pi main.hip -lhiprand
In file included from main.hip:25:
../../Common/hiprand_utils.hpp:28:10: fatal error: 'hiprand/hiprand.h' file not found
28 | #include <hiprand/hiprand.h>
| ^~~~~~~~~~~~~~~~~~~
1 error generated when compiling for gfx1034.
make[2]: *** [Makefile:64: applications_monte_carlo_pi] Error 1
make[2]: Leaving directory '/home/rocm-user/rocm-examples/Applications/monte_carlo_pi'
make[1]: *** [Makefile:41: monte_carlo_pi] Error 2
make[1]: Leaving directory '/home/rocm-user/rocm-examples/Applications'
make: *** [Makefile:34: Applications] Error 2For potential difference between Unix Makefile and CMake, I also got the error: Illegal instruction detected in the manually building with make:
❯ make
make -C Applications
make[1]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications'
make -C bitonic_sort
make[2]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications/bitonic_sort'
make[2]: 'applications_bitonic_sort' is up to date.
make[2]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications/bitonic_sort'
make -C convolution
make[2]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications/convolution'
make[2]: 'applications_convolution' is up to date.
make[2]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications/convolution'
make -C floyd_warshall
make[2]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications/floyd_warshall'
make[2]: 'applications_floyd_warshall' is up to date.
make[2]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications/floyd_warshall'
make -C histogram
make[2]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications/histogram'
make[2]: 'applications_histogram' is up to date.
make[2]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications/histogram'
make -C prefix_sum
make[2]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications/prefix_sum'
make[2]: 'applications_prefix_sum' is up to date.
make[2]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications/prefix_sum'
make -C monte_carlo_pi
make[2]: Entering directory '/home/cr0c0dile/Documents/rocm-examples/Applications/monte_carlo_pi'
/opt/rocm/bin/hipcc -std=c++17 -Wall -Wextra -I ../../Common -isystem /opt/rocm/include -isystem /opt/rocm/include -I ../../Common -D__HIP_PLATFORM_AMD__ -L /opt/rocm/lib -o applications_monte_carlo_pi main.hip -lhiprand
/opt/rocm/bin/rocm_agent_enumerator:95: SyntaxWarning: invalid escape sequence '\w'
@staticVars(search_name=re.compile("gfx[0-9a-fA-F]+(:[-+:\w]+)?"))
/opt/rocm/bin/rocm_agent_enumerator:152: SyntaxWarning: invalid escape sequence '\A'
line_search_term = re.compile("\A\s+Name:\s+(amdgcn-amd-amdhsa--gfx\d+)")
/opt/rocm/bin/rocm_agent_enumerator:154: SyntaxWarning: invalid escape sequence '\A'
line_search_term = re.compile("\A\s+Name:\s+(gfx\d+)")
/opt/rocm/bin/rocm_agent_enumerator:175: SyntaxWarning: invalid escape sequence '\w'
target_search_term = re.compile("1002:\w+")
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr3 = V_MOV_B32_dpp undef $vgpr3(tied-def 0), $vgpr1, 322, 15, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr3 = V_MOV_B32_dpp undef $vgpr3(tied-def 0), $vgpr1, 322, 15, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr3 = V_MOV_B32_dpp undef $vgpr3(tied-def 0), $vgpr1, 322, 15, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr3 = V_MOV_B32_dpp undef $vgpr3(tied-def 0), $vgpr1, 322, 15, 15, 0, implicit $exec
4 errors generated when compiling for gfx1034.
make[2]: *** [Makefile:64: applications_monte_carlo_pi] Error 1
make[2]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications/monte_carlo_pi'
make[1]: *** [Makefile:41: monte_carlo_pi] Error 2
make[1]: Leaving directory '/home/cr0c0dile/Documents/rocm-examples/Applications'
make: *** [Makefile:34: Applications] Error 2for the installed rocprim version, it is 6.0
❯ sudo pacman -Qi rocprim
Name : rocprim
Version : 6.0.2-1
Description : Header-only library providing HIP parallel primitives
Architecture : any
URL : https://rocm.docs.amd.com/projects/rocPRIM/en/latest/index.html
Licenses : MIT
Groups : None
Provides : None
Depends On : rocm-core hip
Optional Deps : None
Required By : hipcub rocalution rocm-hip-sdk rocsparse rocthrust
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 3.08 MiB
Packager : Torsten Keßler <tpkessler@archlinux.org>
Build Date : Mon 26 Feb 2024 03:00:29 PM CST
Install Date : Fri 03 May 2024 05:13:04 AM CST
Install Reason : Installed as a dependency for another package
Install Script : No
Validated By : Signaturefor the versions of other rocm package, they are all 6.0
❯ sudo pacman -Qs roc
local/alsa-card-profiles 1:1.0.5-1
Low-latency audio/video router and processor - ALSA card profiles
local/amd-ucode 20240409.1addd7dc-1
Microcode update image for AMD CPUs
local/comgr 6.0.2-1
Compiler support library for ROCm LLVM
local/ffmpeg 2:6.1.1-7
Complete solution to record, convert and stream audio and video
local/graphite 1:1.3.14-3
reimplementation of the SIL Graphite text processing engine
local/hip-runtime-amd 6.0.2-2
Heterogeneous Interface for Portability ROCm
local/hipblas 6.0.2-1
ROCm BLAS marshalling library
local/hipcub 6.0.2-1
Header-only library on top of rocPRIM or CUB
local/hipfft 6.0.2-1
rocFFT marshalling library.
local/hiprand 6.0.2-1
rocRAND marshalling library
local/hipsolver 6.0.2-1
rocSOLVER marshalling library.
local/hipsparse 6.0.2-1
rocSPARSE marshalling library.
local/hsa-rocr 6.0.2-2
HSA Runtime API and runtime for ROCm
local/hsakmt-roct 6.0.0-2
Radeon Open Compute Thunk Interface
local/libpipewire 1:1.0.5-1
Low-latency audio/video router and processor - client library
local/libvips 8.15.1-5
A fast image processing library with low memory needs
local/libvpl 2.10.2-1
Intel Video Processing Library
local/lsof 4.99.3-2
Lists open files for running Unix processes
local/m4 1.4.19-3
The GNU macro processor
local/pipewire 1:1.0.5-1
Low-latency audio/video router and processor
local/pipewire-alsa 1:1.0.5-1
Low-latency audio/video router and processor - ALSA configuration
local/pipewire-audio 1:1.0.5-1
Low-latency audio/video router and processor - Audio support
local/pipewire-jack 1:1.0.5-1
Low-latency audio/video router and processor - JACK replacement
local/pipewire-pulse 1:1.0.5-1
Low-latency audio/video router and processor - PulseAudio replacement
local/procps-ng 4.0.4-3
Utilities for monitoring your system and its processes
local/psmisc 23.7-1
Miscellaneous procfs tools
local/rccl 6.0.2-1
ROCm Communication Collectives Library
local/rocalution 6.0.2-2
Next generation library for iterative sparse solvers for ROCm platform
local/rocblas 6.0.2-1
Next generation BLAS implementation for ROCm platform
local/rocfft 6.0.2-1
Next generation FFT implementation for ROCm
local/rocm-clang-ocl 6.0.2-1
OpenCL compilation with clang compiler
local/rocm-cmake 6.0.2-1
CMake modules for common build tasks needed for the ROCm software stack
local/rocm-core 6.0.2-2
AMD ROCm core package (version files)
local/rocm-device-libs 6.0.2-1
ROCm Device Libraries
local/rocm-hip-libraries 6.0.2-1
Develop certain applications using HIP and libraries for AMD platforms
local/rocm-hip-runtime 6.0.2-1
Packages to run HIP applications on the AMD platform
local/rocm-hip-sdk 6.0.2-1
Develop applications using HIP and libraries for AMD platforms
local/rocm-language-runtime 6.0.2-1
ROCm runtime
local/rocm-llvm 6.0.2-1
Radeon Open Compute - LLVM toolchain (llvm, clang, lld)
local/rocm-opencl-runtime 6.0.2-1
OpenCL implementation for AMD
local/rocm-opencl-sdk 6.0.2-1
Develop OpenCL-based applications for AMD platforms
local/rocm-smi-lib 6.0.2-1
ROCm System Management Interface Library
local/rocminfo 6.0.2-1
ROCm Application for Reporting System Info
local/rocprim 6.0.2-1
Header-only library providing HIP parallel primitives
local/rocrand 6.0.2-1
Pseudo-random and quasi-random number generator on ROCm
local/rocsolver 6.0.2-1
Subset of LAPACK functionality on the ROCm platform
local/rocsparse 6.0.2-2
BLAS for sparse computation on top of ROCm
local/rocthrust 6.0.2-1
Port of the Thrust parallel algorithm library atop HIP/ROCm
local/roctracer 6.0.2-1
ROCm tracer library for performance tracing
local/spirv-tools 2023.6-1 (vulkan-devel)
API and commands for processing SPIR-V modules
local/vapoursynth R66-2
A video processing framework with the future in mind
local/webrtc-audio-processing-1 1.3-2
AudioProcessing library based on Google's implementation of WebRTCThis is caused by
hipcub::DeviceReduce::Sum, which calls intorocprimon AMD. It was previously caused by macro that wasn't properly defined for these architectures, and has been fixed for 6.0 or 6.1. See ROCm/rocPRIM#452. Could you try with an updated rocPRIM installation? The easiest way to do that is to use therocm/rocm-terminal:6.1docker image.
Snektron
commented
6 months ago I double checked, and it seems that the fix from https://github.com/ROCm/rocPRIM/issues/452 only made its way into ROCm 6.1, and not in 6.0. Unfortunately it seems that the Arch Linux ROCm package is not yet updated to 6.1, but it should work via docker.
The hiprand issue is caused by that hiprand and the other dependencies are not automatically installed in the rocm/rocm-terminal image. You need to install it manually with sudo apt update && sudo apt install hiprand rocrand hipcub rocprim. For your reference, here is the complete list of commands that I'm executing to get a build for your GPU:
$ docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/rocm-terminal:6.1 bash
rocm-user@0b983450b199:~$ sudo apt update && sudo apt install hiprand rocrand hipcub rocprim
# [ ... lots of apt stuff ]
rocm-user@0b983450b199:~$ git clone https://github.com/ROCm/rocm-examples.git
Cloning into 'rocm-examples'...
# [ ... git stuff ]
rocm-user@0b983450b199:~$ cd rocm-examples/Applications/monte_carlo_pi/
rocm-user@0b983450b199:~/rocm-examples/Applications/monte_carlo_pi$ make CXXFLAGS=--offload-arch=gfx1034
rocm-user@0b983450b199:~/rocm-examples/Applications/monte_carlo_pi$I need to explicitly pass CXXFLAGS=--offload-arch=gfx1034 because I don't actually have a RX 6500, but this should be automatically detected for your system if you just use make.
On a side note, the CMake command fails to configure because the Ubuntu image that rocm/rocm-terminal is based on does not come with a recent enough CMake version. In the dockerfiles included with the project, which are also based on rocm/rocm-terminal, we download a more recent version of CMake manually. I'd suggest you to just use these dockerfiles, but they aren't yet updated to 6.1.
Also, the original issue description lists that you have a Radeon VII, but the output from rocminfo --support lists that you have an RX 6500. I think something is not correct there...
Please let me know if this helps.
Gardene-el
commented
6 months ago Thanks a lot, this helps, the building is well after installed the packages needed. About the original description, i'm sorry for that, there is not a option for RX6500 and it is a required option. I've noted this in Additional Information.
Problem Description
Build steps and log: ❯ git clone https://github.com/ROCm/rocm-examples.git
❯ cd rocm-examples ❯ cmake -S . -B build
❯ cmake --build build
Operating System
Arch Linux
CPU
AMD Ryzen 5 5600 6-Core Processor
GPU
AMD Radeon VII
ROCm Version
ROCm 6.0.0
ROCm Component
No response
Steps to Reproduce
git clone https://github.com/ROCm/rocm-examples.git cd rocm-examples cmake -S . -B build cmake --build build
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
Additional Information
Sorry that the above gpu list doesn't show my gpu, i have to pick a fake one instead. Below infomation is about my real gpu. ❯ echo "GPU:" && /opt/rocm/bin/rocminfo | grep -E "^\s*(Name|Marketing Name)"; GPU: Name: AMD Ryzen 5 5600 6-Core Processor
Marketing Name: AMD Ryzen 5 5600 6-Core Processor
Name: gfx1034
Marketing Name: AMD Radeon RX 6500 XT
Name: amdgcn-amd-amdhsa--gfx1034