navi14 (gfx1012): git apply can not find file patch/22.rocblas-ninja-1.patch

Environment

Hardware	description
GPU	RX 5500 XT
CPU	Ryzen 7 5800X

Software	version
OS	Linux Mint 20.3 (Ubuntu focal 20.04 base) under 5.8.0-48-generic x86_64 kernel
ROCm	4.3
Python	3.8.10

What is the expected behavior

I am trying to build ROCm for my navi14 GPU. Dependencies and environment are installed and set. I am following the instructions from navi14/REAME.md.

What actually happens

When executing bash navi14/22.rocblas.sh the script exits with error because git apply can not find file patch/22.rocblas-ninja-1.patch. I looked in the repo and the file is no longer there since commit 7759bdbfe91d5e163e91ce5fc92e025995476e8f. I am not sure how to proceed from here.

How to reproduce

It is not clear in the README: I ran bash navi14/22.rocblas.sh before running any other script because it is the first recommendation, before the list of 10 scripts to run. Do I need to run it in order as well?

I upgrade navi10 documents and scripts to the latest version - ROCm-5.2.3.

And my suggestion is try HSA_OVERRIDE_GFX_VERSION=10.3.0, which needn't re-compile sources.

The documents is based on the rocm-build, if you want to build ROCm from zero, you need run all of scripts from rocm-build one-by-one. If you just want to get navi10 supporting, you can install original ROCm first and re-compile related components.

So I should try installing the ROCm 5.2.3 package instead first? I noticed they moved the install documentation to a new site since 5.x. The requirements seem to be the same as for 4.3. Is the amdgpu stack a requirement for ROCm? Its install step is there in the new documentation.

In my environment, rx5700xt need amdgpu-5.2.3 to prevent PCIe atomic requirement issue.

If there is kfd atomic problem in dmesg, you can try latest amdgpu dkms. And ubuntu-20.04.4 upgraded kernel-5.15.0 which need amdgpu-dkms-5.2.3. ROCm-4.3 may use the legal rocm-dkms, which is likely equals amdgpu-dkms on ROCm-5+.

So my suggestion is using latest amdgpu-dkms to prevent these issues.

Okay I've set up the new 5.2.3 environment. The issue now is the navi14 rocblas patch seems to be failing:

|====|
|SLOW|
|====|
~/code/rocm-build/build/rocblas ~/code/rocm-build/build/rocblas
HEAD is now at da90ed36 Merge pull request #1520 from benjaminulmer/release/rocm-rel-5.2
error: patch failed: Tensile/Common.py:201
error: Tensile/Common.py: patch does not apply
error: patch failed: Tensile/TensileCreateLibrary.py:238
error: Tensile/TensileCreateLibrary.py: patch does not apply

I manually checked both files (Tensile/Common.py and Tensile/TensileCreateLibrary.py) and indeed the patches don't apply.

I believe everything is correctly set up on my end. Is navi14/22.tensile-gfx1012-1.patch outdated, perhaps?

Yes, I forget to update gfx1012 patch. I had synced the gfx1012 patch with gfx1010, please try a again.

So I am trying to compile everything running every script. I could not succeed with just the scripts mentioned under navi14/README.md because they're producing dependency issues and breaking rocm packages. (That's bad, right?)

I compiled everything successfully until miopen. First, the compiler complained about lacking "boost_filesystem" so I installed libboost-filesystem1.71-dev with its dependencies. Then the compiler succeeded, but not the linker. Right now I'm getting this error:

/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_filesystem.a(operations.o): relocation R_X86_64_PC32 against symbol `_ZSt7nothrow@@GLIBCXX_3.4' can not be used when making a shared object; recompile with -fPIC

I tried editing the 35.miopen.sh script file to add the CMAKE flag -DCMAKE_POSITION_INDEPENDENT_CODE=ON but it didn't work. Below you can see clang++ passing the -fPIC argument. I don't know why the compiler fails. Any help?

Here's the full log

➜ LC_ALL=C bash 35.miopen.sh
/usr/local/pip install cget
Requirement already satisfied: cget in /usr/local/lib/python3.8/site-packages (0.2.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/site-packages (from cget) (8.1.3)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.8/site-packages (from cget) (1.16.0)
Downloading https://github.com/pfultz2/rocm-recipes/archive/HEAD.tar.gz

Extracting archive /usr/local/cget/build/tmp-db34b79fc90945efb662d8aa100b70b1/HEAD.tar.gz ...
CMake Warning (dev) in CMakeLists.txt:
  No project() command is present.  The top-level CMakeLists.txt file must
  contain a literal, direct call to the project() command.  Add a line of
  code such as

    project(ProjectName)

  near the top of the file, but after cmake_minimum_required().

  CMake is pretending there is a "project(Project)" command on the first
  line.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- The C compiler identification is Clang 14.0.0
-- The CXX compiler identification is Clang 14.0.0
-- Check for working C compiler: /opt/rocm/llvm/bin/clang
-- Check for working C compiler: /opt/rocm/llvm/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/rocm/llvm/bin/clang++
-- Check for working CXX compiler: /opt/rocm/llvm/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.16)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING
    CGET_CMAKE_DIR
    CGET_CMAKE_ORIGINAL_SOURCE_FILE

-- Build files have been written to: /usr/local/cget/build/tmp-db34b79fc90945efb662d8aa100b70b1/build
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.78
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.78/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.78/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.65
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.65/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.65/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.58
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.58/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.58/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.72
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.72/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.72/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.79
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.79/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.79/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//bzip2
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//bzip2/build.cmake
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//bzip2/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf/build.cmake
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//libressl
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//libressl/build.cmake
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//libressl/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//pcre
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//pcre/requirements.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//pcre/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc/local
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc/local/CMakeLists.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//sqlite3
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//sqlite3/3.17
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//sqlite3/3.17/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//zlib
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//zlib/package.txt
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/share//cmake
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/share//cmake/cget-recipe-utils
-- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/share//cmake/cget-recipe-utils/cget-recipe-utils-config.cmake
Successfully installed pfultz2/rocm-recipes
~/code/rocm-build/build/miopen ~/code/rocm-build/build/miopen
-- hip::amdhip64 is SHARED_LIBRARY
-- Build with HIP 5.2.22304
-- Hip compiler flags:  -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -x hip  -isystem /opt/rocm-5.2.3/include -isystem /opt/rocm-5.2.3/llvm/lib/clang/14.0.0/include/..  -D__HIP_PLATFORM_HCC__=1 -D__HIP_PLATFORM_AMD__=1  -isystem /opt/rocm-5.2.3/include -isystem /opt/rocm/include --hip-link  
-- OpenCL compiler: /opt/rocm/bin/clang-ocl
-- hip::amdhip64 is SHARED_LIBRARY
-- Build with rocblas
-- HIP backend selected.
-- clang-offload-bundler found: /opt/rocm/llvm/bin/clang-offload-bundler
-- AMDGCN assembler: /opt/rocm/llvm/bin/clang
-- Build without miopentensile
-- Build without miopengemm
-- Build with comgr 2.4.0
-- HALF_INCLUDE_DIR: /opt/rocm/include
RPM version 4.14.2.1
-- rocm-cmake: Set license file to /home/tom/code/rocm/MIOpen/LICENSE.txt.
-- Clang tidy not found
-- Clang tidy checks: *,-abseil-*,-altera-struct-pack-align,-altera-unroll-loops,-android-cloexec-fopen,-bugprone-exception-escape,-bugprone-macro-parentheses,-bugprone-narrowing-conversions,-cert-dcl37-c,-cert-dcl51-cpp,-cert-env33-c,-cert-msc30-c,-cert-msc32-c,-cert-msc50-cpp,-cert-msc51-cpp,-clang-analyzer-alpha.core.CastToStruct,-clang-analyzer-optin.performance.Padding,-clang-diagnostic-deprecated-declarations,-clang-diagnostic-extern-c-compat,-clang-diagnostic-unused-command-line-argument,-cppcoreguidelines-avoid-c-arrays,-cppcoreguidelines-avoid-magic-numbers,-cppcoreguidelines-explicit-virtual-functions,-cppcoreguidelines-init-variables,-cppcoreguidelines-macro-usage,-cppcoreguidelines-narrowing-conversions,-cppcoreguidelines-non-private-member-variables-in-classes,-cppcoreguidelines-prefer-member-initializer,-cppcoreguidelines-pro-bounds-array-to-pointer-decay,-cppcoreguidelines-pro-bounds-constant-array-index,-cppcoreguidelines-pro-bounds-pointer-arithmetic,-cppcoreguidelines-pro-type-member-init,-cppcoreguidelines-pro-type-reinterpret-cast,-cppcoreguidelines-pro-type-union-access,-cppcoreguidelines-pro-type-vararg,-cppcoreguidelines-special-member-functions,-fuchsia-*,-google-explicit-constructor,-google-readability-braces-around-statements,-google-readability-todo,-google-runtime-int,-google-runtime-references,-hicpp-avoid-c-arrays,-hicpp-braces-around-statements,-hicpp-explicit-conversions,-hicpp-named-parameter,-hicpp-no-array-decay,-hicpp-signed-bitwise,-hicpp-special-member-functions,-hicpp-uppercase-literal-suffix,-hicpp-use-auto,-hicpp-use-equals-default,-hicpp-use-override,-hicpp-vararg,-llvm-else-after-return,-llvm-header-guard,-llvm-include-order,-llvmlibc-callee-namespace,-llvmlibc-implementation-in-namespace,-llvmlibc-restrict-system-libc-headers,-llvm-qualified-auto,-misc-misplaced-const,-misc-non-private-member-variables-in-classes,-misc-no-recursion,-modernize-avoid-bind,-modernize-avoid-c-arrays,-modernize-pass-by-value,-modernize-use-auto,-modernize-use-default-member-init,-modernize-use-equals-default,-modernize-use-trailing-return-type,-modernize-use-transparent-functors,-performance-unnecessary-value-param,-readability-braces-around-statements,-readability-convert-member-functions-to-static,-readability-else-after-return,-readability-function-cognitive-complexity,-readability-isolate-declaration,-readability-magic-numbers,-readability-named-parameter,-readability-qualified-auto,-readability-redundant-string-init,-readability-uppercase-literal-suffix,-modernize-use-override,-readability-non-const-parameter
-- Could NOT find LATEX (missing: LATEX_COMPILER) 
Latex builder not found. Latex builder is required only for building the PDF documentation for MIOpen and is not necessary for building the library, or any other components. To build PDF documentation run make in /home/tom/code/rocm/MIOpen/doc/pdf, once a latex builder is installed.
-- MIOpen_VERSION= 2.17.0
-- CMAKE_BUILD_TYPE= Release
-- Librt: /usr/lib/x86_64-linux-gnu/librt.so
-- Backward Compatible Sym Link Created for include directories
-- rocminfo utility: /opt/rocm/bin/rocminfo
-- MIOPEN_NO_GPU FALSE
-- MIOPEN_TEST_GFX900 OFF
-- MIOPEN_TEST_GFX906 OFF
-- MIOPEN_TEST_GFX908 OFF
-- MIOPEN_TEST_GFX90A OFF
-- MIOPEN_TEST_GFX1030 ON
-- MIOPEN_TEST_GPU_XNACK_ENABLED OFF
-- MIOPEN_TEST_GPU_DETECTION_FAILED FALSE
-- SKIP_TESTS: 
-- SKIP_ALL_EXCEPT_TESTS: 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/tom/code/rocm-build/build/miopen
[1/3] Linking CXX shared library lib/libMIOpen.so.1.0.50200
FAILED: lib/libMIOpen.so.1.0.50200 
`: && /opt/rocm/llvm/bin/clang++ -fPIC -O3 -DNDEBUG -s  -pthread -shared -Wl,-soname,libMIOpen.so.1 -o lib/libMIOpen.so.1.0.50200 src/sqlite/CMakeFiles/sqlite_memvfs.dir/memvfs.cpp.o src/CMakeFiles/MIOpen.dir/__/kernel.cpp.o src/CMakeFiles/MIOpen.dir/__/kernel_includes.cpp.o src/CMakeFiles/MIOpen.dir/activ/problem_description.cpp.o src/CMakeFiles/MIOpen.dir/activ_api.cpp.o src/CMakeFiles/MIOpen.dir/batch_norm.cpp.o src/CMakeFiles/MIOpen.dir/batch_norm_api.cpp.o src/CMakeFiles/MIOpen.dir/batchnorm/problem_description.cpp.o src/CMakeFiles/MIOpen.dir/buffer_info.cpp.o src/CMakeFiles/MIOpen.dir/check_numerics.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/gcn_asm_1x1u.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/gcn_asm_1x1u_ss.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/gcn_asm_1x1u_us.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/gen_x_w_y_pad.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/impl_gemm.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/impl_gemm_dynamic.cpp.o src/CMakeFiles/MIOpen.dir/conv/invokers/ocl_wrw_rdc.cpp.o src/CMakeFiles/MIOpen.dir/conv/problem_description.cpp.o src/CMakeFiles/MIOpen.dir/conv_algo_name.cpp.o src/CMakeFiles/MIOpen.dir/convolution.cpp.o src/CMakeFiles/MIOpen.dir/convolution_api.cpp.o src/CMakeFiles/MIOpen.dir/ctc.cpp.o src/CMakeFiles/MIOpen.dir/ctc_api.cpp.o src/CMakeFiles/MIOpen.dir/db.cpp.o src/CMakeFiles/MIOpen.dir/db_record.cpp.o src/CMakeFiles/MIOpen.dir/dropout.cpp.o src/CMakeFiles/MIOpen.dir/dropout_api.cpp.o src/CMakeFiles/MIOpen.dir/execution_context.cpp.o src/CMakeFiles/MIOpen.dir/expanduser.cpp.o src/CMakeFiles/MIOpen.dir/find_controls.cpp.o src/CMakeFiles/MIOpen.dir/find_db.cpp.o src/CMakeFiles/MIOpen.dir/fused_api.cpp.o src/CMakeFiles/MIOpen.dir/fusion.cpp.o src/CMakeFiles/MIOpen.dir/handle_api.cpp.o src/CMakeFiles/MIOpen.dir/invoker_cache.cpp.o src/CMakeFiles/MIOpen.dir/kernel_build_params.cpp.o src/CMakeFiles/MIOpen.dir/kernel_warnings.cpp.o src/CMakeFiles/MIOpen.dir/load_file.cpp.o src/CMakeFiles/MIOpen.dir/lock_file.cpp.o src/CMakeFiles/MIOpen.dir/logger.cpp.o src/CMakeFiles/MIOpen.dir/lrn_api.cpp.o src/CMakeFiles/MIOpen.dir/md_graph.cpp.o src/CMakeFiles/MIOpen.dir/mdg_expr.cpp.o src/CMakeFiles/MIOpen.dir/op_args.cpp.o src/CMakeFiles/MIOpen.dir/operator.cpp.o src/CMakeFiles/MIOpen.dir/pooling/problem_description.cpp.o src/CMakeFiles/MIOpen.dir/pooling_api.cpp.o src/CMakeFiles/MIOpen.dir/problem_description.cpp.o src/CMakeFiles/MIOpen.dir/ramdb.cpp.o src/CMakeFiles/MIOpen.dir/readonlyramdb.cpp.o src/CMakeFiles/MIOpen.dir/reducetensor.cpp.o src/CMakeFiles/MIOpen.dir/reducetensor_api.cpp.o src/CMakeFiles/MIOpen.dir/rnn.cpp.o src/CMakeFiles/MIOpen.dir/rnn_api.cpp.o src/CMakeFiles/MIOpen.dir/softmax_api.cpp.o src/CMakeFiles/MIOpen.dir/solver.cpp.o src/CMakeFiles/MIOpen.dir/solver/activ/bwd_0.cpp.o src/CMakeFiles/MIOpen.dir/solver/activ/bwd_1.cpp.o src/CMakeFiles/MIOpen.dir/solver/activ/fwd_0.cpp.o src/CMakeFiles/MIOpen.dir/solver/activ/fwd_1.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/backward_per_activation.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/backward_spatial_multiple.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/backward_spatial_single.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/forward_inference.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/forward_per_activation.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/forward_spatial_multiple.cpp.o src/CMakeFiles/MIOpen.dir/solver/batchnorm/forward_spatial_single.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_1x1u.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_1x1u_bias_activ.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_1x1u_stride2.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_3x3u.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_5x10u2v2b1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_5x10u2v2f1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_7x7c3h224w224k64u2v2p3q3f1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_dir_BwdWrW1x1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_dir_BwdWrW3x3.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_bwd_v4r1_dynamic.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_gtc_bwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_gtc_bwd_nhwc.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_gtc_fwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_gtc_fwd_nhwc.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_gtc_perf_config.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_gtc_wrw_nhwc.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_v4r1_dynamic.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_wrw_gtc_dynamic_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_asm_implicit_gemm_wrw_v4r1_dynamic.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_bin_wino3x3U.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_bin_winoRxS.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ck_igemm_fwd_v6r1_dlops_nchw.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_direct_naive_conv.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_direct_naive_conv_bwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_direct_naive_conv_fwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_direct_naive_conv_wrw.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_bwd_v1r1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_bwd_v1r1_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_bwd_v4r1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_bwd_v4r1_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_fwd_v4r1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_fwd_v4r4.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_fwd_v4r4_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_fwd_v4r4_xdlops_padded_gemm.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_fwd_v4r5_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_nonxdlops_common.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_wrw_v4r4.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_wrw_v4r4_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_wrw_v4r4_xdlops_padded_gemm.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_hip_implicit_gemm_xdlops_common.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_mlir_igemm_bwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_mlir_igemm_bwd_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_mlir_igemm_fwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_mlir_igemm_fwd_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_mlir_igemm_wrw.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_mlir_igemm_wrw_xdlops.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_MP_bidirectional_winograd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_multipass_wino3x3WrW.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2D_bwdWrW_1x1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2D_bwdWrW_2.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2D_bwdWrW_53.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2D11x11.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2Dfwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2Dfwd_exhaustive_search.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2Dfwd1x1.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_ocl_dir2Dfwdgen.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_winoRxS_f2x3.cpp.o src/CMakeFiles/MIOpen.dir/solver/conv_winoRxS_f3x2.cpp.o src/CMakeFiles/MIOpen.dir/solver/fft.cpp.o src/CMakeFiles/MIOpen.dir/solver/gemm.cpp.o src/CMakeFiles/MIOpen.dir/solver/gemm_bwd.cpp.o src/CMakeFiles/MIOpen.dir/solver/gemm_common.cpp.o src/CMakeFiles/MIOpen.dir/solver/gemm_wrw.cpp.o src/CMakeFiles/MIOpen.dir/solver/pooling/forward2d.cpp.o src/CMakeFiles/MIOpen.dir/solver/pooling/forwardNd.cpp.o src/CMakeFiles/MIOpen.dir/subbuffers.cpp.o src/CMakeFiles/MIOpen.dir/target_properties.cpp.o src/CMakeFiles/MIOpen.dir/temp_file.cpp.o src/CMakeFiles/MIOpen.dir/tensor.cpp.o src/CMakeFiles/MIOpen.dir/tensor_api.cpp.o src/CMakeFiles/MIOpen.dir/tmp_dir.cpp.o src/CMakeFiles/MIOpen.dir/binary_cache.cpp.o src/CMakeFiles/MIOpen.dir/md5.cpp.o src/CMakeFiles/MIOpen.dir/sqlite_db.cpp.o src/CMakeFiles/MIOpen.dir/kern_db.cpp.o src/CMakeFiles/MIOpen.dir/bz2.cpp.o src/CMakeFiles/MIOpen.dir/activ.cpp.o src/CMakeFiles/MIOpen.dir/kernel_cache.cpp.o src/CMakeFiles/MIOpen.dir/lrn.cpp.o src/CMakeFiles/MIOpen.dir/mlo_dir_conv.cpp.o src/CMakeFiles/MIOpen.dir/exec_utils.cpp.o src/CMakeFiles/MIOpen.dir/ocl/activ_ocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/batchnormocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/convolutionocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/lrn_ocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/mloNeuron.cpp.o src/CMakeFiles/MIOpen.dir/ocl/mloNorm.cpp.o src/CMakeFiles/MIOpen.dir/ocl/mloPooling.cpp.o src/CMakeFiles/MIOpen.dir/ocl/pooling_ocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/tensorocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/softmaxocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/rnnocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/utilocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/ctcocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/dropoutocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/gcn_asm_utils.cpp.o src/CMakeFiles/MIOpen.dir/ocl/rnn_util_ocl.cpp.o src/CMakeFiles/MIOpen.dir/hip/hip_build_utils.cpp.o src/CMakeFiles/MIOpen.dir/hip/batched_transpose_sol.cpp.o src/CMakeFiles/MIOpen.dir/hip/general_tensor_reorder_sol.cpp.o src/CMakeFiles/MIOpen.dir/pooling.cpp.o src/CMakeFiles/MIOpen.dir/ocl/fusionopconvocl.cpp.o src/CMakeFiles/MIOpen.dir/ocl/fusionopbiasbnactivocl.cpp.o src/CMakeFiles/MIOpen.dir/__/db_path.cpp.o src/CMakeFiles/MIOpen.dir/gemm_v2.cpp.o src/CMakeFiles/MIOpen.dir/miopengemm.cpp.o src/CMakeFiles/MIOpen.dir/hip/hiperrors.cpp.o src/CMakeFiles/MIOpen.dir/hip/handlehip.cpp.o src/CMakeFiles/MIOpen.dir/hipoc/hipoc_kernel.cpp.o src/CMakeFiles/MIOpen.dir/hipoc/hipoc_program.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_0.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_1.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_2.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_3.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_4.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_5.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_6.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_7.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_8.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_9.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_10.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_11.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_12.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_13.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_14.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_15.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_16.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_17.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_18.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_19.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_20.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_21.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_22.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_23.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_24.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_25.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_26.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_27.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_28.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_29.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_30.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_31.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_32.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_33.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_34.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_35.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_36.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_37.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_38.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_39.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_40.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_41.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_42.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_43.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_44.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_45.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_46.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_47.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_48.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_49.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_50.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_51.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_52.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_53.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_54.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_55.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_56.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_57.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_58.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_59.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_60.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_61.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_62.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_63.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_64.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_65.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_66.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_67.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_68.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_69.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_70.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_71.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_72.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_73.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_74.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_75.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_76.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_77.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_78.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_79.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_80.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_81.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_82.cpp.o src/CMakeFiles/MIOpen.dir/__/inlined_kernels/batch_83.cpp.o src/CMakeFiles/MIOpen.dir/comgr.cpp.o  -Wl,-rpath,/opt/rocm/lib:/opt/rocm-5.2.3/lib:  -lpthread  /usr/lib/x86_64-linux-gnu/libbz2.so  /opt/rocm/lib/libamd_comgr.so.2.4.50200  /opt/rocm-5.2.3/lib/librocblas.so.0.1.50200  /usr/lib/x86_64-linux-gnu/libboost_filesystem.a  -Wl,--version-script=/home/tom/code/rocm-build/build/miopen/src/lib.def  -Wl,--exclude-libs,ALL  -lsqlite3  -lm  -ldl  -lz  -lpthread  /usr/lib/x86_64-linux-gnu/librt.so  /usr/lib/x86_64-linux-gnu/libbz2.so  -lsqlite3  -lm  -ldl  -lz  /usr/lib/x86_64-linux-gnu/librt.so  --hip-link  --offload-arch=gfx1012  /opt/rocm-5.2.3/llvm/lib/clang/14.0.0/lib/linux/libclang_rt.builtins-x86_64.a  /opt/rocm/lib/libamdhip64.so.5.2.50100 && :`
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_filesystem.a(operations.o): relocation R_X86_64_PC32 against symbol `_ZSt7nothrow@@GLIBCXX_3.4' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

The MIOpen need a customized boost-filesystem, we need sudo cmake -P $ROCM_GIT_DIR/MIOpen/install_deps.cmake --prefix /usr/local to do that, it will download boost-system-1.71 and use AMD customized cmake files to build boost-filesystem.

I guess MIOpen maybe need a static library boost-filesystem.

I see now. I removed my distro's related libboost*-dev packages and re-ran the script. The cget script exits successfully:

Successfully installed pfultz2/rocm-recipes

However the missing dependency error remains. There is in fact no /usr/include/boost path present in my filesystem.

If it's correctly building boost-filesystem, I'm not sure where it's being installed. (Edit: locate finds nothing of value)

Full log

➜ LC_ALL=C bash 35.miopen.sh /usr/local/pip install cget Requirement already satisfied: cget in /usr/local/lib/python3.8/site-packages (0.2.0) Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/site-packages (from cget) (8.1.3) Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.8/site-packages (from cget) (1.16.0) Downloading https://github.com/pfultz2/rocm-recipes/archive/HEAD.tar.gz Extracting archive /usr/local/cget/build/tmp-f8d141f9a42f42f2b9bba7f78f4ba02e/HEAD.tar.gz ... CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as project(ProjectName) near the top of the file, but after cmake_minimum_required(). CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it. -- The C compiler identification is Clang 14.0.0 -- The CXX compiler identification is Clang 14.0.0 -- Check for working C compiler: /opt/rocm/llvm/bin/clang -- Check for working C compiler: /opt/rocm/llvm/bin/clang -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /opt/rocm/llvm/bin/clang++ -- Check for working CXX compiler: /opt/rocm/llvm/bin/clang++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Warning (dev) in CMakeLists.txt: No cmake_minimum_required command is present. A line of code such as cmake_minimum_required(VERSION 3.16) should be added at the top of the file. The version specified may be lower if you wish to support older CMake versions for this project. For more information run "cmake --help-policy CMP0000". This warning is for project developers. Use -Wno-dev to suppress it. -- Configuring done -- Generating done CMake Warning: Manually-specified variables were not used by the project: BUILD_TESTING CGET_CMAKE_DIR CGET_CMAKE_ORIGINAL_SOURCE_FILE -- Build files have been written to: /usr/local/cget/build/tmp-f8d141f9a42f42f2b9bba7f78f4ba02e/build Install the project... -- Install configuration: "Release" -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.78 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.78/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.78/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.65 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.65/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.65/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.58 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.58/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.58/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.72 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.72/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.72/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.79 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.79/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//boost/1.79/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//bzip2 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//bzip2/build.cmake -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//bzip2/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf/build.cmake -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//google/protobuf/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//libressl -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//libressl/build.cmake -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//libressl/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//pcre -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//pcre/requirements.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//pcre/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc/local -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc/local/CMakeLists.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//require-hcc/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//sqlite3 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//sqlite3/3.17 -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//sqlite3/3.17/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//zlib -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/etc/cget/recipes//zlib/package.txt -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/share//cmake -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/share//cmake/cget-recipe-utils -- Installing: /usr/local/cget/pkg/pfultz2__rocm-recipes/install/share//cmake/cget-recipe-utils/cget-recipe-utils-config.cmake Successfully installed pfultz2/rocm-recipes ~/code/rocm-build/build/miopen ~/code/rocm-build/build/miopen -- hip::amdhip64 is SHARED_LIBRARY -- Build with HIP 5.2.22304 -- Hip compiler flags: -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -x hip -isystem /opt/rocm-5.2.3/include -isystem /opt/rocm-5.2.3/llvm/lib/clang/14.0.0/include/.. -D__HIP_PLATFORM_HCC__=1 -D__HIP_PLATFORM_AMD__=1 -isystem /opt/rocm-5.2.3/include -isystem /opt/rocm/include --hip-link -- OpenCL compiler: /opt/rocm/bin/clang-ocl -- hip::amdhip64 is SHARED_LIBRARY -- Build with rocblas -- HIP backend selected. -- clang-offload-bundler found: /opt/rocm/llvm/bin/clang-offload-bundler -- AMDGCN assembler: /opt/rocm/llvm/bin/clang -- Build without miopentensile -- Build without miopengemm -- Build with comgr 2.4.0 CMake Error at /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindBoost.cmake:1636 (file): file STRINGS file "/usr/include/boost/version.hpp" cannot be read. Call Stack (most recent call first): CMakeLists.txt:388 (find_package) CMake Warning at /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindBoost.cmake:851 (message): Imported targets and dependency information not available for Boost version 0.0.0 (all versions older than 1.33) Call Stack (most recent call first): /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindBoost.cmake:1283 (_Boost_COMPONENT_DEPENDENCIES) /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindBoost.cmake:1921 (_Boost_MISSING_DEPENDENCIES) CMakeLists.txt:388 (find_package) CMake Error at /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message): Could NOT find Boost (missing: filesystem) (found version "0.0.0") Call Stack (most recent call first): /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE) /home/tom/.local/cmake-3.16.8-Linux-x86_64/share/cmake-3.16/Modules/FindBoost.cmake:2179 (find_package_handle_standard_args) CMakeLists.txt:388 (find_package) -- Configuring incomplete, errors occurred! See also "/home/tom/code/rocm-build/build/miopen/CMakeFiles/CMakeOutput.log". See also "/home/tom/code/rocm-build/build/miopen/CMakeFiles/CMakeError.log".

There is no compiling and installing logs of boost. My suggestion is check why boost cmake didnot execute.

There is no compiling and installing logs of boost. My suggestion is check why boost cmake didnot execute.

Well, to start: ginger.amd.com is down. That's the domain address where the builder wants to get boost from. It's the same for the other dependencies (bzip2, sqlite3, zlib).

I don't know why the logs don't show a connection/download issue though.

I searched for the boost package hash included in the file linked above. It matches the official 1.72.0 boost package file, which means installing that version of boost-system and boost-filesystem in my system could be enough to go. My Mint version is based off of Ubuntu 20.04 which only provides libboost 1.71.0, which is possibly why my previous attempt was failing.

I managed to run 35.miopen.sh successfully. I had to manually build and install the boost libraries myself because I could not get the recipe-based build system to work. Fortunately there were no conflicts there.

I failed to run 43.rocgdb.sh. This one's pretty memory intensive but I'm ok. The linker is throwing a lot of undefined reference errors, most of them relating to python objects. I skipped this step anyway because I don't think I have a need for gdb.

Right now I can't build rocalution. Not sure why I'm getting these GPU target related errors. What do you think?

➜ LC_ALL=C bash 54.rocalution.sh 
~/code/rocm-build/build/rocalution ~/code/rocm-build/build/rocalution
HEAD is now at 92181e1 Update CHANGELOG.md
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS) 
-- Could NOT find MPI (missing: MPI_CXX_FOUND) 
-- MPI not found. Compiling WITHOUT MPI support.
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
RPM version 4.14.2.1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/tom/code/rocm-build/build/rocalution
[1/126] Building HIPCC object src/CMak...ion_hip_generated_hip_matrix_csr.cpp.o
FAILED: src/CMakeFiles/rocalution_hip.dir/base/hip/rocalution_hip_generated_hip_matrix_csr.cpp.o 
cd /home/tom/code/rocm-build/build/rocalution/src/CMakeFiles/rocalution_hip.dir/base/hip && /home/tom/.local/cmake-3.16.8-Linux-x86_64/bin/cmake -E make_directory /home/tom/code/rocm-build/build/rocalution/src/CMakeFiles/rocalution_hip.dir/base/hip/. && /home/tom/.local/cmake-3.16.8-Linux-x86_64/bin/cmake -D verbose:BOOL=OFF -D build_configuration:STRING=RELEASE -D generated_file:STRING=/home/tom/code/rocm-build/build/rocalution/src/CMakeFiles/rocalution_hip.dir/base/hip/./rocalution_hip_generated_hip_matrix_csr.cpp.o -P /home/tom/code/rocm-build/build/rocalution/src/CMakeFiles/rocalution_hip.dir/base/hip/rocalution_hip_generated_hip_matrix_csr.cpp.o.cmake
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 323, 12, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 323, 12, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 323, 12, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 323, 12, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 323, 12, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 322, 10, 15, 0, implicit $exec
error: Illegal instruction detected: Invalid dpp_ctrl value: broadcasts are not supported on GFX10+
renamable $vgpr4 = V_MOV_B32_dpp undef $vgpr4(tied-def 0), $vgpr3, 323, 12, 15, 0, implicit $exec
13 errors generated when compiling for gfx1012.
CMake Error at rocalution_hip_generated_hip_matrix_csr.cpp.o.cmake:200 (message):
  Error generating file
  /home/tom/code/rocm-build/build/rocalution/src/CMakeFiles/rocalution_hip.dir/base/hip/./rocalution_hip_generated_hip_matrix_csr.cpp.o

ninja: build stopped: subcommand failed.

At this version, rocalution is not necessary for pytorch or tensorflow, so we can skip it.

It will be fixed on rocm-5.3 or later.

So between 61.amdmigraphx.sh and 75.hipify.sh, which ones are needed for pytorch?

While trying to run 61.amdmigraphx.sh I got this error and I solved as explained there. But I also got the following ONNX error:

[115/813] Building CXX object src/onnx...es/migraphx_onnx.dir/onnx_parser.cpp.o
FAILED: src/onnx/CMakeFiles/migraphx_onnx.dir/onnx_parser.cpp.o 
/opt/rocm/llvm/bin/clang++  -DHAS_HALF_V1 -Dmigraphx_onnx_EXPORTS -I/home/tom/code/rocm/AMDMIGraphX/src/onnx/include -I/home/tom/code/rocm/AMDMIGraphX/src/include -Isrc/include -isystem src/onnx -O3 -DNDEBUG -fPIC   -std=c++17 -Wall -Wextra -Wcomment -Wendif-labels -Wformat -Winit-self -Wreturn-type -Wsequence-point -Wswitch -Wtrigraphs -Wundef -Wuninitialized -Wunreachable-code -Wunused -Wno-sign-compare -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-conversion -Wno-double-promotion -Wno-exit-time-destructors -Wno-extra-semi -Wno-extra-semi-stmt -Wno-float-conversion -Wno-gnu-anonymous-struct -Wno-gnu-zero-variadic-macro-arguments -Wno-missing-prototypes -Wno-nested-anon-types -Wno-padded -Wno-shorten-64-to-32 -Wno-sign-conversion -Wno-unused-command-line-argument -Wno-weak-vtables -Wno-c99-extensions -MD -MT src/onnx/CMakeFiles/migraphx_onnx.dir/onnx_parser.cpp.o -MF src/onnx/CMakeFiles/migraphx_onnx.dir/onnx_parser.cpp.o.d -o src/onnx/CMakeFiles/migraphx_onnx.dir/onnx_parser.cpp.o -c /home/tom/code/rocm/AMDMIGraphX/src/onnx/onnx_parser.cpp
/home/tom/code/rocm/AMDMIGraphX/src/onnx/onnx_parser.cpp:365:58: error: no member named 'at' in 'google::protobuf::RepeatedPtrField<onnx_for_migraphx::StringStringEntryProto>'
        const std::string& data_file = t.external_data().at(0).value();
                                       ~~~~~~~~~~~~~~~~~ ^
1 error generated.
[132/813] Building CXX object src/CMak...s/migraphx_op_quantizelinear_hpp.cpp.o
ninja: build stopped: subcommand failed.

The compiler/linker seems to be using libprotobuf v3.2.0 (automatically fetched and built). But the ONNX source is referencing protobuf code written for a much later version (v3.15+). Should I change the ONNX / AMDMIGraphX dependency to use a newer libprotobuf? What do you think?

miopen and rocsolver is the required components, the components after rocsolver is not used by tensorflow or pytorch now.

amdmigraphx is a onnx runtime like onnx-runtime of microsoft, actually I havent test amdgraphx for a while.

I went ahead and began compiling pytorch. Everything went well until:

[6378/6695] Linking CXX executable bin/hip_generator_test
FAILED: bin/hip_generator_test 
: && /usr/bin/c++  -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG  -rdynamic   -o bin/hip_generator_test  -Wl,-rpath,/home/tom/code/pytorch-rocm/build/lib:/opt/rocm/miopen/lib:/opt/rocm/hip/lib:/opt/rocm/roctracer/lib:/opt/rocm-5.2.3/lib:/opt/rocm/lib:  lib/libgtest_main.a  -Wl,--no-as-needed,"/home/tom/code/pytorch-rocm/build/lib/libtorch.so" -Wl,--as-needed  -Wl,--no-as-needed,"/home/tom/code/pytorch-rocm/build/lib/libtorch_cpu.so" -Wl,--as-needed  lib/libprotobuf.a  -Wl,--no-as-needed,"/home/tom/code/pytorch-rocm/build/lib/libtorch_hip.so" -Wl,--as-needed  lib/libc10_hip.so  lib/libc10.so  /opt/rocm/miopen/lib/libMIOpen.so  /opt/rocm/hip/lib/libamdhip64.so  /opt/rocm/roctracer/lib/libroctx64.so  /opt/rocm-5.2.3/lib/librocblas.so.0.1.50203  /opt/rocm/lib/libamdhip64.so.5.2.50203  /opt/rocm/llvm/lib/clang/14.0.0/lib/linux/libclang_rt.builtins-x86_64.a  /opt/rocm-5.2.3/lib/libhipfft.so  /opt/rocm-5.2.3/lib/libhiprand.so.1.1.50203  /opt/rocm-5.2.3/lib/libhipsparse.so.0.1.50203  lib/libgtest.a  -pthread  -Wl,-rpath-link,/opt/rocm-5.2.3/lib && :

The linker then complains that librocm_smi64.so.5 is missing. I checked and the rocm-smi-lib builder only installs a /opt/rocm/rocm_smi/lib/librocm_smi64.so.2.8 file. Not sure why there is a version mismatch here.

Which version branch of pytorch do you use? I havn't met this issue before.

Sorry, I was once again victim of Mint. For some reason after reboot the /opt/rocm symlink gets lost, which was causing several issues.

I finally managed to compile and install pytorch 1.12.1. Now to test it on my GPU. 😄

Thanks so much for your help! It's been a challenge and I couldn't have pulled through on my own.

Sadly, my GPU is locking up when I run this simple test:

import torch
torch.tensor([1., 2.], device='cuda')

Even when torch.cuda.is_available() returns True.

Many things could have gone wrong here. Not sure if there's much I can do now.

Edit: I just realized this test fails differently if I run python from the rocm-build environment (i.e. if I execute source env.sh first). torch.cuda.is_available() returns "hipErrorNoBinaryForGpu: Unable to find code object for all current devices!"

The hipErrorNoBinaryForGpu said there cannot find gfx1012 target in somewhere. As we know, ROCm need compiled for specific target so it can support multi arch.

HIP_DB=0xF AMD_LOG_LEVEL=4 python3 test.py then logs will show where we are hanged. But it is not very clearly to show the reason of issue. We should do some guess by ourselves.

You could do some test before run pytorch, as pytorch is the last piece of the whole MI tasks. E.G. /opt/rocm/bin/rocminfo, /opt/rcom/bin/rocm-smi.

What's causing hipErrorNoBinaryForGpu is definitely HSA_OVERRIDE_GFX_VERSION=10.3.0.

Without HSA override, rocminfo is showing expected output. rocm-smi as well.

Here are logs for the command you suggested (video hanged for several seconds until I killed the process):

➜ HIP_DB=0xF AMD_LOG_LEVEL=4 python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.tensor([1., 2.], device='cuda')
:3:rocdevice.cpp            :416 : 2619441943 us: 8473 : [tid:0x7f1d24b76740] Initializing HSA stack.
:3:comgrctx.cpp             :33  : 2619470142 us: 8473 : [tid:0x7f1d24b76740] Loading COMGR library.
:3:rocdevice.cpp            :205 : 2619470179 us: 8473 : [tid:0x7f1d24b76740] Numa selects cpu agent[0]=0x3ef4880(fine=0x590a300,coarse=0x59a3bc0) for gpu agent=0x59a25d0
:3:rocdevice.cpp            :1610: 2619470411 us: 8473 : [tid:0x7f1d24b76740] HMM support: 0, xnack: 0, direct host access: 0

:4:rocdevice.cpp            :1918: 2619470432 us: 8473 : [tid:0x7f1d24b76740] Allocate hsa host memory 0x7f1c65901000, size 0x28
:4:rocdevice.cpp            :1918: 2619470663 us: 8473 : [tid:0x7f1d24b76740] Allocate hsa host memory 0x7f1c63800000, size 0x101000
:4:rocdevice.cpp            :1918: 2619470969 us: 8473 : [tid:0x7f1d24b76740] Allocate hsa host memory 0x7f1c63600000, size 0x101000
:4:rocdevice.cpp            :2054: 2619471186 us: 8473 : [tid:0x7f1d24b76740] Allocate hsa device memory 0x7f1c63200000, size 0x100000
:4:runtime.cpp              :83  : 2619471190 us: 8473 : [tid:0x7f1d24b76740] init
:3:hip_context.cpp          :50  : 2619471192 us: 8473 : [tid:0x7f1d24b76740] Direct Dispatch: 1
:3:hip_device_runtime.cpp   :517 : 2619477825 us: 8473 : [tid:0x7f1d24b76740] hipGetDeviceCount: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :515 : 2619478399 us: 8473 : [tid:0x7f1d24b76740] hipGetDeviceCount ( 0x7ffe7fb02508 )
:3:hip_device_runtime.cpp   :517 : 2619478403 us: 8473 : [tid:0x7f1d24b76740] hipGetDeviceCount: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :500 : 2619480622 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice ( 0x7ffe7fb028e4 )
:3:hip_device_runtime.cpp   :508 : 2619480628 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :500 : 2619484974 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice ( 0x7ffe7fb01c24 )
:3:hip_device_runtime.cpp   :508 : 2619484979 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :500 : 2619485220 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice ( 0x7ffe7fb01b04 )
:3:hip_device_runtime.cpp   :508 : 2619485223 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :500 : 2619485712 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice ( 0x7ffe7fb01a94 )
:3:hip_device_runtime.cpp   :508 : 2619485716 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice: Returned hipSuccess : 
:3:hip_error.cpp            :27  : 2619485732 us: 8473 : [tid:0x7f1d24b76740] hipGetLastError (  )
:3:hip_memory.cpp           :493 : 2619486843 us: 8473 : [tid:0x7f1d24b76740] hipMalloc ( 0x7ffe7fb01548, 2097152 )
:4:rocdevice.cpp            :2054: 2619487647 us: 8473 : [tid:0x7f1d24b76740] Allocate hsa device memory 0x7f1c62e00000, size 0x200000
:3:rocdevice.cpp            :2093: 2619487653 us: 8473 : [tid:0x7f1d24b76740] device=0x59c4d20, freeMem_ = 0xfee00000
:3:hip_memory.cpp           :495 : 2619487657 us: 8473 : [tid:0x7f1d24b76740] hipMalloc: Returned hipSuccess : 0x7f1c62e00000: duration: 814 us
:3:hip_device_runtime.cpp   :530 : 2619487667 us: 8473 : [tid:0x7f1d24b76740] hipSetDevice ( 0 )
:3:hip_device_runtime.cpp   :535 : 2619487670 us: 8473 : [tid:0x7f1d24b76740] hipSetDevice: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :530 : 2619487672 us: 8473 : [tid:0x7f1d24b76740] hipSetDevice ( 0 )
:3:hip_device_runtime.cpp   :535 : 2619487676 us: 8473 : [tid:0x7f1d24b76740] hipSetDevice: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :500 : 2619490538 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice ( 0x7ffe7fb01bb4 )
:3:hip_device_runtime.cpp   :508 : 2619490542 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice: Returned hipSuccess : 
:3:hip_device_runtime.cpp   :500 : 2619490546 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice ( 0x7ffe7fb01c1c )
:3:hip_device_runtime.cpp   :508 : 2619490548 us: 8473 : [tid:0x7f1d24b76740] hipGetDevice: Returned hipSuccess : 
:3:hip_memory.cpp           :566 : 2619490557 us: 8473 : [tid:0x7f1d24b76740] hipMemcpyWithStream ( 0x7f1c62e00000, 0x172eac0, 8, hipMemcpyHostToDevice, stream:<null> )
:3:rocdevice.cpp            :2682: 2619490562 us: 8473 : [tid:0x7f1d24b76740] number of allocated hardware queues with low priority: 0, with normal priority: 0, with high priority: 0, maximum per priority is: 4
:3:rocdevice.cpp            :2756: 2619494118 us: 8473 : [tid:0x7f1d24b76740] created hardware queue 0x7f1c646a8000 with size 4096 with priority 1, cooperative: 0
:4:rocdevice.cpp            :1918: 2619494193 us: 8473 : [tid:0x7f1d24b76740] Allocate hsa host memory 0x7f1c63500000, size 0x80000
:3:devprogram.cpp           :2675: 2619870723 us: 8473 : [tid:0x7f1d24b76740] Using Code Object V4.
:3:devprogram.cpp           :2978: 2619873278 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_streamOpsWrite
:3:devprogram.cpp           :2978: 2619873284 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_fillImage
:3:devprogram.cpp           :2978: 2619873287 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_fillBufferAligned2D
:3:devprogram.cpp           :2978: 2619873289 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyImageToBuffer
:3:devprogram.cpp           :2978: 2619873291 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyImage1DA
:3:devprogram.cpp           :2978: 2619873292 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_fillBufferAligned
:3:devprogram.cpp           :2978: 2619873294 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyImage
:3:devprogram.cpp           :2978: 2619873296 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_streamOpsWait
:3:devprogram.cpp           :2978: 2619873298 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyBufferRectAligned
:3:devprogram.cpp           :2978: 2619873300 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyBufferRect
:3:devprogram.cpp           :2978: 2619873303 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyBufferAligned
:3:devprogram.cpp           :2978: 2619873305 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyBufferToImage
:3:devprogram.cpp           :2978: 2619873307 us: 8473 : [tid:0x7f1d24b76740] For Init/Fini: Kernel Name: __amd_rocclr_copyBuffer
:4:rocmemory.cpp            :940 : 2619873379 us: 8473 : [tid:0x7f1d24b76740] Locking to pool 0x59a3bc0, size 0x1000, HostPtr = 0x63ea000, DevPtr = 0x7f1c639b7000
:4:command.cpp              :339 : 2619873645 us: 8473 : [tid:0x7f1d24b76740] command is enqueued: 0x67ef600
:4:rocblit.cpp              :723 : 2619873653 us: 8473 : [tid:0x7f1d24b76740] HSA Async Copy completion_signal=0x7f1c65905600
:4:rocvirtual.cpp           :523 : 2619874017 us: 8473 : [tid:0x7f1d24b76740] Host wait on completion_signal=0x7f1c65905600
:3:rocvirtual.hpp           :61  : 2619874020 us: 8473 : [tid:0x7f1d24b76740] Host active wait for Signal = (0x7f1c65905600) for -1 ns
[1]    8473 terminated  HIP_DB=0xF AMD_LOG_LEVEL=4 python3

I have to say that I had met this hanging on mem sync things, and after I move my gpu from the lower PCIe slot to other higher PCIe slot, it is ok, then.

I don't know whether there is cpu or motherboard or hard driver 's pass number limit or south bridge things. But ther are two PCIe slots on the motherboard, one ok, one hanging.

So my suggestion is changing other PCIe slot, and try again. And good luck. That is AMD and ROCm things, you know.

I reinstalled amdgpu-dkms and that did the trick. The python console test is now working:

>>> import torch
>>> torch.cuda.is_available()
True
>>> torch.tensor([1., 2.], device='cuda')
tensor([1., 2.], device='cuda:0')

Now to see what I can actually do with navi14. Thank you very much @xuhuisheng once more.

I think there was a Tensile related problem during rocblas build, because this happens when running stable-diffusion:

rocBLAS error: Cannot read /opt/rocm-5.2.3/lib/rocblas/library/TensileLibrary.dat: No such file or directory

I checked and that path has no TensileLibrary.dat file even within the generated rocblas .deb package. Here's the full rocblas build log:

➜ LC_ALL=C bash navi14/22.rocblas.sh
|====|
|SLOW|
|====|
~/code/rocm-build/build/rocblas ~/code/rocm-build/build/rocblas
HEAD is now at da90ed36 Merge pull request #1520 from benjaminulmer/release/rocm-rel-5.2
-- Use hip-clang to build for amdgpu backend
-- OS detected is linuxmint
/usr/bin/python3 -m venv /home/tom/code/rocm-build/build/rocblas/virtualenv --system-site-packages --clear
/home/tom/code/rocm-build/build/rocblas/virtualenv/bin/python3 -m pip install /home/tom/code/rocm/Tensile
Processing /home/tom/code/rocm/Tensile
Requirement already satisfied: msgpack in /usr/lib/python3/dist-packages (from Tensile==4.33.0) (0.6.2)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.8/site-packages (from Tensile==4.33.0) (6.0)
Building wheels for collected packages: Tensile
  Building wheel for Tensile (setup.py): started
  Building wheel for Tensile (setup.py): finished with status 'done'
  Created wheel for Tensile: filename=Tensile-4.33.0-py3-none-any.whl size=4565810 sha256=ebe1ad4114a2bd3acfba0afe6f305bcae73f488b65f577c3d06f2603df6974f7
  Stored in directory: /tmp/pip-ephem-wheel-cache-n6c9crui/wheels/eb/1f/6a/441a8ef306808840995ea43d3f369f4c78ddd773ccd8840ff1
Successfully built Tensile
Installing collected packages: Tensile
Successfully installed Tensile-4.33.0
-- using local Tensile from /home/tom/code/rocm/Tensile, copied to 
-- Adding /home/tom/code/rocm-build/build/rocblas/virtualenv to CMAKE_PREFIX_PATH
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- Using AMDGPU_TARGETS: gfx1012
-- Tensile script: /home/tom/code/rocm-build/build/rocblas/virtualenv/lib/python3.8/site-packages/Tensile/bin/TensileCreateLibrary
-- Tensile_CREATE_COMMAND: /home/tom/code/rocm-build/build/rocblas/virtualenv/lib/python3.8/site-packages/Tensile/bin/TensileCreateLibrary;--merge-files;--separate-architectures;--no-short-file-names;--no-library-print-debug;--code-object-version=V3;--cxx-compiler=hipcc;--library-format=msgpack;--architecture=gfx1012;/home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full;/home/tom/code/rocm-build/build/rocblas/Tensile;HIP
-- Tensile_MANIFEST_FILE_PATH: /home/tom/code/rocm-build/build/rocblas/Tensile/library/TensileManifest.txt
'/home/tom/code/rocm-build/build/rocblas/virtualenv/lib/python3.8/site-packages/Tensile/bin/TensileCreateLibrary' '--merge-files' '--separate-architectures' '--no-short-file-names' '--no-library-print-debug' '--code-object-version=V3' '--cxx-compiler=hipcc' '--library-format=msgpack' '--architecture=gfx1012' '/home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full' '/home/tom/code/rocm-build/build/rocblas/Tensile' 'HIP' '--generate-manifest-and-exit'
Message pack python library not detected. Must use YAML backend instead.

################################################################################
# Tensile Create Library
# Detected local GPU with ISA: gfx1012
               cap gfx000 gfx1012 
   HasMFMA_bf16_1k      0       0 
        HasAddLshl      0       1 
      HasAtomicAdd      0       0 
   HasCodeObjectV3      0       1 
    HasDirectToLds      0       1 
     HasExplicitCO      0       1 
     HasExplicitNC      0       1 
         HasLshlOr      0       1 
           HasMFMA      0       0 
         HasSMulHi      0       1 
        MaxLgkmcnt      1       1 
          MaxVmcnt      0       1 
      SupportedISA      0       1 
   SupportedSource      1       1 
    v_dot2_f32_f16      0       1 
   v_dot2c_f32_f16      0       1 
         v_fma_f16      0       1 
        v_fmac_f16      0       0 
         v_mac_f16      0       0 
      v_pk_fma_f16      0       1 
     v_pk_fmac_f16      0       0 
         v_fma_f32      0       1 
     v_fma_mix_f32      0       1 
        v_fmac_f32      0       1 
         v_mac_f32      0       1 
     v_mad_mix_f32      0       0 
       HasMFMA_f64      0       0 
     v_dot4_i32_i8      0       0 
    v_dot4c_i32_i8      0       1 
ArchAccUnifiedRegs      0       0 
    CMPXWritesSGPR      1       0 
          HasAccCD      0       0 
        HasEccHalf      0       0 
         HasWave32      0       1 
     SeparateVscnt      0       1 
  Waitcnt0Disabled      0       0 
# Found  hipcc version 5.2.21153-02187ecf
# CodeObjectVersion from TensileCreateLibrary: V3
# CxxCompiler       from TensileCreateLibrary: hipcc
# Architecture      from TensileCreateLibrary: gfx1012
# LibraryFormat     from TensileCreateLibrary: msgpack
# LibraryLogicFiles:
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bjlk_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_DB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_BBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_BSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_BSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_HSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_SB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_BjlkC_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_HSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_BSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_HB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_4xi8BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_HB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_I8II_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_HBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bljk_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_DB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_BjlkC_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_HSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_HSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_BBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_BSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_I8II_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_I8II_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_I8II_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_BSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_BSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_BjlkC_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bjlk_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_BBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_HBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_BjlkC_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_HBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_4xi8BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_BjlkC_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_SB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_HB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bjlk_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_HSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_BBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_DB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_HB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_I8II_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_HBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_I8II_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_HB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bljk_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_I8II_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_DB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_BjlkC_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_HSS_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_HBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_DB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_I8II_BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_SB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_4xi8BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_SB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_BBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_HB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_BjlkC_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_BBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_HB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_BjlkC_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_HBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_HSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_BjlkC_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_SB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_SB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_SB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_4xi8BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_BSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bjlk_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bljk_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_HSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_BjlkC_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_SB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_BBH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_4xi8BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_DB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_4xi8BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_4xi8BH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_BjlkC_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_HB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_HBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_4xi8BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bjlk_CB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_BjlkC_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_AlikC_Bljk_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_DB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_ZB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Ailk_Bljk_BSS_BH_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_HBH.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_CB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_ZB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bljk_DB_GB.yaml
#   /home/tom/code/rocm/rocBLAS/library/src/blas3/Tensile/Logic/asm_full/hip_Cijk_Alik_Bjlk_BBH_GB.yaml
Reading logic files: Launching 16 threads...
Reading logic files: Done.
[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||] 100% (0.4 secs elapsed)
RPM version 4.14.2.1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/tom/code/rocm-build/build/rocblas
[0/2] Re-checking globbed directories...
[23/23] Creating library symlink libra...rocblas.so.0 library/src/librocblas.so
[0/2] Re-checking globbed directories...
[1/2] Run CPack packaging tool...
CPack: Create package using DEB
CPack: Install projects
CPack: - Install project: rocblas []
CPack: -   Install component: runtime
CPack: -   Install component: devel
CPack: Create package
CPack: - package: /home/tom/code/rocm-build/build/rocblas/rocblas-dev_2.44.0.50200-65_amd64.deb generated.
CPack: - package: /home/tom/code/rocm-build/build/rocblas/rocblas_2.44.0.50200-65_amd64.deb generated.
(Reading database ... 432778 files and directories currently installed.)
Preparing to unpack rocblas-dev_2.44.0.50200-65_amd64.deb ...
Unpacking rocblas-dev (2.44.0.50200-65) over (2.44.0.50200-65) ...
Preparing to unpack rocblas_2.44.0.50200-65_amd64.deb ...
rm: cannot remove '/etc/ld.so.conf.d/rocblas.conf': No such file or directory
Unpacking rocblas (2.44.0.50200-65) over (2.44.0.50200-65) ...
Setting up rocblas (2.44.0.50200-65) ...
Setting up rocblas-dev (2.44.0.50200-65) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
elapse : 61s
~/code/rocm-build/build/rocblas

Notice the Message pack python library not detected. Must use YAML backend instead. line. I have all of libmsgpack-dev, python3-msgpack and python3-msgpack-numpy installed.

What do you think?

Seems I didn't sync navi14/22.rocblas.sh with gfx803, I changed Tensile_SEPARATE_ARCHITECTURES to OFF, the dat file appeared. I had pushed updated scripts to git, please try.

Ok now it's attempting to generate the libraries, but is failing:

[1/24] Generating Tensile Libraries
FAILED: Tensile/library/TensileLibrary.dat Tensile/library/Kernels.so-000-gfx1012.hsaco

Also a python error at the end complaining for lack of msgpack symbols, which is strange because the python module is supposedly installed in Tensile's virtualenv. Attaching full log.

rocblas.txt

xuhuisheng / rocm-build