search

ROCm / rocm_smi_lib

ROCm SMI LIB
https://rocm.docs.amd.com/projects/rocm_smi_lib/en/latest/
MIT License
123 stars 50 forks source link

[Issue]: Cannot set fan speed for 7900XTX #164

Open Looong01 opened 7 months ago

Looong01 commented 7 months ago

Problem Description

When I try to use rocm-smi or amd-smi to set fan speed, I find that only 6700XT successfully but 7900XTX failed. And rocm-smi command returns this:

image

It cannot display the marketing name of 7900XTX

OS: NAME="Ubuntu" VERSION="22.04.4 LTS (Jammy Jellyfish)"

CPU: model name : Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz

GPU: Name: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz Marketing Name: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz Name: gfx1100
Marketing Name:
Name: amdgcn-amd-amdhsa--gfx1100
Name: gfx1100
Marketing Name: AMD Radeon RX 6700 XT
Name: amdgcn-amd-amdhsa--gfx1030

(base) loong@ubuntu-server-220403:~/Codes/CV$ sudo amd-smi set -g 0 -f 80%

        ******WARNING******

        Operating your AMD GPU outside of official AMD specifications or outside of
        factory settings, including but not limited to the conducting of overclocking,
        over-volting or under-volting (including use of this interface software,
        even if such software has been directly or indirectly provided by AMD or otherwise
        affiliated in any way with AMD), may cause damage to your AMD GPU, system components
        and/or result in system failure, as well as cause other problems.
        DAMAGES CAUSED BY USE OF YOUR AMD GPU OUTSIDE OF OFFICIAL AMD SPECIFICATIONS OR
        OUTSIDE OF FACTORY SETTINGS ARE NOT COVERED UNDER ANY AMD PRODUCT WARRANTY AND
        MAY NOT BE COVERED BY YOUR BOARD OR SYSTEM MANUFACTURER'S WARRANTY.
        Please use this utility with caution.

Do you accept these terms? [y/n] y amdsmi.amdsmi_exception.AmdSmiLibraryException: Error code: 2 | AMDSMI_STATUS_NOT_SUPPORTED - Feature not supported

The above exception was the direct cause of the following exception:

ValueError: Unable to set fan speed 204 on GPU ID: 0 BDF:0000:03:00.0 (base) loong@ubuntu-server-220403:~/Codes/CV$ sudo amd-smi set -g 1 -f 80%

        ******WARNING******

        Operating your AMD GPU outside of official AMD specifications or outside of
        factory settings, including but not limited to the conducting of overclocking,
        over-volting or under-volting (including use of this interface software,
        even if such software has been directly or indirectly provided by AMD or otherwise
        affiliated in any way with AMD), may cause damage to your AMD GPU, system components
        and/or result in system failure, as well as cause other problems.
        DAMAGES CAUSED BY USE OF YOUR AMD GPU OUTSIDE OF OFFICIAL AMD SPECIFICATIONS OR
        OUTSIDE OF FACTORY SETTINGS ARE NOT COVERED UNDER ANY AMD PRODUCT WARRANTY AND
        MAY NOT BE COVERED BY YOUR BOARD OR SYSTEM MANUFACTURER'S WARRANTY.
        Please use this utility with caution.

Do you accept these terms? [y/n] y GPU: 1 FAN: Successfully set fan speed 204

Operating System

Ubuntu 22.04.4 LTS (Jammy Jellyfish)

CPU

Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz

GPU

AMD Radeon RX 7900 XTX

ROCm Version

ROCm 6.0.0

ROCm Component

amdsmi, rocm_smi_lib

Steps to Reproduce

(base) loong@ubuntu-server-220403:~/Codes/CV$ sudo amd-smi set -g 0 -f 80%

        ******WARNING******

        Operating your AMD GPU outside of official AMD specifications or outside of
        factory settings, including but not limited to the conducting of overclocking,
        over-volting or under-volting (including use of this interface software,
        even if such software has been directly or indirectly provided by AMD or otherwise
        affiliated in any way with AMD), may cause damage to your AMD GPU, system components
        and/or result in system failure, as well as cause other problems.
        DAMAGES CAUSED BY USE OF YOUR AMD GPU OUTSIDE OF OFFICIAL AMD SPECIFICATIONS OR
        OUTSIDE OF FACTORY SETTINGS ARE NOT COVERED UNDER ANY AMD PRODUCT WARRANTY AND
        MAY NOT BE COVERED BY YOUR BOARD OR SYSTEM MANUFACTURER'S WARRANTY.
        Please use this utility with caution.

Do you accept these terms? [y/n] y amdsmi.amdsmi_exception.AmdSmiLibraryException: Error code: 2 | AMDSMI_STATUS_NOT_SUPPORTED - Feature not supported

The above exception was the direct cause of the following exception:

ValueError: Unable to set fan speed 204 on GPU ID: 0 BDF:0000:03:00.0 (base) loong@ubuntu-server-220403:~/Codes/CV$ sudo amd-smi set -g 1 -f 80%

        ******WARNING******

        Operating your AMD GPU outside of official AMD specifications or outside of
        factory settings, including but not limited to the conducting of overclocking,
        over-volting or under-volting (including use of this interface software,
        even if such software has been directly or indirectly provided by AMD or otherwise
        affiliated in any way with AMD), may cause damage to your AMD GPU, system components
        and/or result in system failure, as well as cause other problems.
        DAMAGES CAUSED BY USE OF YOUR AMD GPU OUTSIDE OF OFFICIAL AMD SPECIFICATIONS OR
        OUTSIDE OF FACTORY SETTINGS ARE NOT COVERED UNDER ANY AMD PRODUCT WARRANTY AND
        MAY NOT BE COVERED BY YOUR BOARD OR SYSTEM MANUFACTURER'S WARRANTY.
        Please use this utility with caution.

Do you accept these terms? [y/n] y GPU: 1 FAN: Successfully set fan speed 204

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

ROCk module is loaded =====================
HSA System Attributes
=====================
Runtime Version: 1.1 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE
System Endianness: LITTLE
Mwaitx: DISABLED DMAbuf Support: YES

==========
HSA Agents
==========

Agent 1

Name: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz Uuid: CPU-XX
Marketing Name: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz Vendor Name: CPU
Feature: None specified
Profile: FULL_PROFILE
Float Round Mode: NEAR
Max Queue Number: 0(0x0)
Queue Min Size: 0(0x0)
Queue Max Size: 0(0x0)
Queue Type: MULTI
Node: 0
Device Type: CPU
Cache Info:
L1: 32768(0x8000) KB
Chip ID: 0(0x0)
ASIC Revision: 0(0x0)
Cacheline Size: 64(0x40)
Max Clock Freq. (MHz): 4700
BDFID: 0
Internal Node ID: 0
Compute Unit: 8
SIMDs per CU: 0
Shader Engines: 0
Shader Arrs. per Eng.: 0
WatchPts on Addr. Ranges:1
Features: None Pool Info:
Pool 1
Segment: GLOBAL; FLAGS: FINE GRAINED
Size: 65775456(0x3eba760) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: TRUE
Pool 2
Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED Size: 65775456(0x3eba760) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: TRUE
Pool 3
Segment: GLOBAL; FLAGS: COARSE GRAINED
Size: 65775456(0x3eba760) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: TRUE
ISA Info:

Agent 2

Name: gfx1100
Uuid: GPU-85631fd855c9cea1
Marketing Name:
Vendor Name: AMD
Feature: KERNEL_DISPATCH
Profile: BASE_PROFILE
Float Round Mode: NEAR
Max Queue Number: 128(0x80)
Queue Min Size: 64(0x40)
Queue Max Size: 131072(0x20000)
Queue Type: MULTI
Node: 1
Device Type: GPU
Cache Info:
L1: 32(0x20) KB
L2: 6144(0x1800) KB
L3: 98304(0x18000) KB
Chip ID: 29772(0x744c)
ASIC Revision: 0(0x0)
Cacheline Size: 64(0x40)
Max Clock Freq. (MHz): 2482
BDFID: 768
Internal Node ID: 1
Compute Unit: 96
SIMDs per CU: 2
Shader Engines: 6
Shader Arrs. per Eng.: 2
WatchPts on Addr. Ranges:4
Coherent Host Access: FALSE
Features: KERNEL_DISPATCH Fast F16 Operation: TRUE
Wavefront Size: 32(0x20)
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension: x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Max Waves Per CU: 32(0x20)
Max Work-item Per CU: 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension: x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
Max fbarriers/Workgrp: 32
Packet Processor uCode:: 550
SDMA engine uCode:: 19
IOMMU Support:: None
Pool Info:
Pool 1
Segment: GLOBAL; FLAGS: COARSE GRAINED
Size: 25149440(0x17fc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 2
Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 25149440(0x17fc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 3
Segment: GROUP
Size: 64(0x40) KB
Allocatable: FALSE
Alloc Granule: 0KB
Alloc Alignment: 0KB
Accessible by all: FALSE
ISA Info:
ISA 1
Name: amdgcn-amd-amdhsa--gfx1100
Machine Models: HSA_MACHINE_MODEL_LARGE
Profiles: HSA_PROFILE_BASE
Default Rounding Mode: NEAR
Default Rounding Mode: NEAR
Fast f16: TRUE
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension: x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension: x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
FBarrier Max Size: 32

Agent 3

Name: gfx1100
Uuid: GPU-XX
Marketing Name: AMD Radeon RX 6700 XT
Vendor Name: AMD
Feature: KERNEL_DISPATCH
Profile: BASE_PROFILE
Float Round Mode: NEAR
Max Queue Number: 128(0x80)
Queue Min Size: 64(0x40)
Queue Max Size: 131072(0x20000)
Queue Type: MULTI
Node: 2
Device Type: GPU
Cache Info:
L1: 16(0x10) KB
L2: 3072(0xc00) KB
L3: 98304(0x18000) KB
Chip ID: 29663(0x73df)
ASIC Revision: 0(0x0)
Cacheline Size: 64(0x40)
Max Clock Freq. (MHz): 2855
BDFID: 2304
Internal Node ID: 2
Compute Unit: 40
SIMDs per CU: 2
Shader Engines: 2
Shader Arrs. per Eng.: 2
WatchPts on Addr. Ranges:4
Coherent Host Access: FALSE
Features: KERNEL_DISPATCH Fast F16 Operation: TRUE
Wavefront Size: 32(0x20)
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension: x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Max Waves Per CU: 32(0x20)
Max Work-item Per CU: 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension: x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
Max fbarriers/Workgrp: 32
Packet Processor uCode:: 116
SDMA engine uCode:: 80
IOMMU Support:: None
Pool Info:
Pool 1
Segment: GLOBAL; FLAGS: COARSE GRAINED
Size: 12566528(0xbfc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 2
Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 12566528(0xbfc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 3
Segment: GROUP
Size: 64(0x40) KB
Allocatable: FALSE
Alloc Granule: 0KB
Alloc Alignment: 0KB
Accessible by all: FALSE
ISA Info:
ISA 1
Name: amdgcn-amd-amdhsa--gfx1100
Machine Models: HSA_MACHINE_MODEL_LARGE
Profiles: HSA_PROFILE_BASE
Default Rounding Mode: NEAR
Default Rounding Mode: NEAR
Fast f16: TRUE
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension: x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension: x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
FBarrier Max Size: 32
Done

Additional Information

No response

dmitrii-galantsev commented 7 months ago

you can try and trace it to the sysfs file and then edit it directly: sudo strace --trace=file --successful-only /opt/rocm/bin/rocm-smi --setfan 100% 2>&1 | grep sys/class

This will return a bunch of stuff and point you to something like: /sys/class/drm/card1/device/hwmon/hwmon2/pwm1

See if you can set these manually.

I'm expecting that this is a driver issue and amdsmi/rocmsmi can't do anything about it.

P.S. You can also run it with sudo RSMI_LOGGING=1 /opt/rocm/bin/rocm-smi --setfan 100%. it will populate /var/log/rocm_smi_lib/ROCm-SMI-lib.log. =3 will additionally print to console.

Looong01 commented 7 months ago

I try this:

sudo strace --trace=file --successful-only /opt/rocm/bin/rocm-smi --setfan 100% 2>&1 | grep sys/class

and it returns:

$ sudo strace --trace=file --successful-only /opt/rocm/bin/rocm-smi --setfan 100% 2>&1 | grep sys/class
openat(AT_FDCWD, "/sys/class/drm", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/vendor", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/vendor", O_RDONLY) = 4
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 5
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/name", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/temp1_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/temp1_label", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/temp2_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/temp2_label", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/temp3_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/temp3_label", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/in0_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/in0_label", O_RDONLY) = 6
openat(AT_FDCWD, "/sys/class/drm/card0/device/drm", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 5
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/unique_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/unique_id", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/vendor", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/vendor", O_RDONLY) = 5
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/name", O_RDONLY) = 7
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/temp1_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/temp1_label", O_RDONLY) = 7
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/temp2_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/temp2_label", O_RDONLY) = 7
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/temp3_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/temp3_label", O_RDONLY) = 7
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/in0_label", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/in0_label", O_RDONLY) = 7
openat(AT_FDCWD, "/sys/class/drm/card1/device/drm", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
readlink("/sys/class/drm/card0", "../../devices/pci0000:00/0000:00"..., 512) = 86
readlink("/sys/class/drm/card1", "../../devices/pci0000:00/0000:00"..., 512) = 86
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/properties", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/gpu_id", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/name", O_RDONLY) = 6
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", O_RDONLY) = 7
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/properties", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/gpu_id", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/name", O_RDONLY) = 6
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", O_RDONLY) = 7
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/properties", O_RDONLY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/gpu_id", O_RDONLY) = 6
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/1/io_links/0/properties", O_RDONLY) = 7
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
newfstatat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/2/io_links/0/properties", O_RDONLY) = 7
openat(AT_FDCWD, "/sys/class/kfd/kfd/topology/nodes/0/io_links", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/vendor", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/vendor", O_RDONLY) = 9
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/pwm1_max", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/pwm1_max", O_RDONLY) = 9
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/pwm1_enable", {st_mode=S_IFREG|0644, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/pwm1_enable", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 9
newfstatat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/pwm1", {st_mode=S_IFREG|0644, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card0/device/hwmon/hwmon3/pwm1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 9
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/pwm1_max", {st_mode=S_IFREG|0444, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/pwm1_max", O_RDONLY) = 9
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/pwm1_enable", {st_mode=S_IFREG|0644, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/pwm1_enable", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 9
newfstatat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/pwm1", {st_mode=S_IFREG|0644, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "/sys/class/drm/card1/device/hwmon/hwmon4/pwm1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 9
Looong01 commented 7 months ago

I also run sudo RSMI_LOGGING=1 /opt/rocm/bin/rocm-smi --setfan 100%

and the content of log file(/var/log/rocm_smi_lib/ROCm-SMI-lib.log) is:

:/var/log/rocm_smi_lib$ cat ROCm-SMI-lib.log
2024-04-26 08:52:55.460212  [ALWAYS]: =============== ROCM SMI initialize ================
2024-04-26 08:52:55.460279  [INFO]: ====== Gathered system details ============
SYSTEM NAME: Linux
OS DISTRIBUTION: Ubuntu 22.04.4 LTS
NODE NAME: ubuntu-server-220403
RELEASE: 6.5.0-28-generic
VERSION: #29~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  4 14:39:20 UTC 2
MACHINE TYPE: x86_64
DOMAIN: (none)
ENDIANNESS: Little Endian, multi-bit symbols encoded as little endian (LSB first)
ROCM BUILD TYPE: debug
ROCM-SMI-LIB PATH: /opt/rocm-6.0.2/libexec/rocm_smi/../../lib/librocm_smi64.so.6
ROCM-SMI-LIB BUILD DATE: Thu Feb  8 23:54:30 2024
ROCM ENV VARIABLES:
        RSMI_DEBUG_BITFIELD = <undefined>
        RSMI_DEBUG_DRM_ROOT_OVERRIDE = <undefined>
        RSMI_DEBUG_HWMON_ROOT_OVERRIDE = <undefined>
        RSMI_DEBUG_PP_ROOT_OVERRIDE = <undefined>
        RSMI_DEBUG_INFINITE_LOOP = <undefined>
        RSMI_LOGGING = 1
        RSMI_LOGGING (are logs on) = TRUE
        RSMI_DEBUG_ENUM_OVERRIDE = {}
AMD GFX VERSIONS:
        N/A - No AMD devices detected

2024-04-26 08:52:55.460299  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | Discovered a potential of 10 cards |
2024-04-26 08:52:55.460366  [DEBUG]: int amd::smi::get_gpu_id(uint32_t, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/0/gpu_id | Successfully read node #0 for gpu_id | Data (gpu_id) *gpu_id = 0 | return = 0 |
2024-04-26 08:52:55.460415  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/0/properties | Successfully read node #0 for property_name = unique_id | Data (unique_id) * val = 0 | return = 22 |
2024-04-26 08:52:55.460458  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/0/properties | Successfully read node #0 for property_name = location_id | Data (location_id) * val = 0 | return = 0 |
2024-04-26 08:52:55.460563  [DEBUG]: int amd::smi::get_gpu_id(uint32_t, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/1/gpu_id | Successfully read node #1 for gpu_id | Data (gpu_id) *gpu_id = 7642 | return = 0 |
2024-04-26 08:52:55.460647  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/1/properties | Successfully read node #1 for property_name = unique_id | Data (unique_id) * val = 9611561043751456417 | return = 0 |
2024-04-26 08:52:55.460729  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/1/properties | Successfully read node #1 for property_name = location_id | Data (location_id) * val = 768 | return = 0 |
2024-04-26 08:52:55.460821  [DEBUG]: int amd::smi::get_gpu_id(uint32_t, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/2/gpu_id | Successfully read node #2 for gpu_id | Data (gpu_id) *gpu_id = 50916 | return = 0 |
2024-04-26 08:52:55.460902  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/2/properties | Successfully read node #2 for property_name = unique_id | Data (unique_id) * val = 0 | return = 0 |
2024-04-26 08:52:55.460982  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/2/properties | Successfully read node #2 for property_name = location_id | Data (location_id) * val = 2304 | return = 0 |
2024-04-26 08:52:55.460993  [ERROR]: int amd::smi::get_gpu_id(uint32_t, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/3/gpu_id | Issue: Could not read node #3, KFD node was an unsupported node. | return = 1 |
2024-04-26 08:52:55.461001  [ERROR]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/3/properties | Issue: Could not read node #3, KFD node was an unsupported node. | return = 1 |
2024-04-26 08:52:55.461009  [ERROR]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/3/properties | Issue: Could not read node #3, KFD node was an unsupported node. | return = 1 |
2024-04-26 08:52:55.461017  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | Ordered system nodes found = {
[node_id = 2; gpu_id = 50916; unique_id = 0; location_id = 2304],
[node_id = 1; gpu_id = 7642; unique_id = 9611561043751456417; location_id = 768], }
2024-04-26 08:52:55.461038  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card0 is an amdgpu device - TRUE
2024-04-26 08:52:55.461045  [TRACE]: rsmi_status_t rsmi_dev_compute_partition_get(uint32_t, char*, uint32_t)| ======= start =======, dv_ind = 0
2024-04-26 08:52:55.461050  [TRACE]: rsmi_status_t rsmi_dev_unique_id_get(uint32_t, uint64_t*)| ======= start =======
2024-04-26 08:52:55.461054  [TRACE]: void amd::smi::RocmSMI::AddToDeviceList(std::string, uint64_t) | ======= start =======
2024-04-26 08:52:55.461091  [TRACE]: int32_t amd::smi::Monitor::setTempSensorLabelMap() | ======= start =======
2024-04-26 08:52:55.461109  [INFO]: Successfully read SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/temp1_label), returning str = edge
2024-04-26 08:52:55.461113  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp1_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 1 | Data: edge | Returning: 0 |
2024-04-26 08:52:55.461127  [INFO]: Successfully read SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/temp2_label), returning str = junction
2024-04-26 08:52:55.461131  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp2_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 2 | Data: junction | Returning: 0 |
2024-04-26 08:52:55.461145  [INFO]: Successfully read SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/temp3_label), returning str = mem
2024-04-26 08:52:55.461149  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp3_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 3 | Data: mem | Returning: 0 |
2024-04-26 08:52:55.461156  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp4_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 4 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461162  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp5_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 5 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461169  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp6_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 6 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461175  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/temp7_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 7 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461189  [INFO]: Successfully read SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/in0_label), returning str = vddgfx
2024-04-26 08:52:55.461193  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/in0_label | Type: MonitorTypes::kMonVoltLabel | Sensor id: 0 | Data: vddgfx | Returning: 0 |
2024-04-26 08:52:55.461204  [DEBUG]: uint32_t amd::smi::GetDrmRenderMinor(std::string) | Discovered drmRenderMinor = 128 | For drm_path = /sys/class/drm/card0/device/drm |
2024-04-26 08:52:55.461213  [DEBUG]: void amd::smi::RocmSMI::AddToDeviceList(std::string, uint64_t) | Adding to device list dev_name = card0 | path = /sys/class/drm/card0 | bdfid = 18446744073709551615 | card index = 0 |
2024-04-26 08:52:55.461221  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | Ordered system nodes seen in lookup = {
[node_id = 2; gpu_id = 50916; unique_id = 0; location_id = 2304],
[node_id = 1; gpu_id = 7642; unique_id = 9611561043751456417; location_id = 768], }
2024-04-26 08:52:55.461224  [TRACE]: rsmi_status_t rsmi_dev_compute_partition_get(uint32_t, char*, uint32_t)| ======= start =======, dv_ind = 0
2024-04-26 08:52:55.461233  [ERROR]: int amd::smi::Device::openSysfsFileStream(amd::smi::DevInfoTypes, T*, const char*) [with T = std::basic_ifstream<char>] | Issue: File did not exist - SYSFS file (/sys/class/drm/card0/device/current_compute_partition) for DevInfoInfoType (amd::smi::kDevComputePartition), returning 2
2024-04-26 08:52:55.461236  [ERROR]: Could not read device info string for DevInfoType (amd::smi::kDevComputePartition), returning 2
2024-04-26 08:52:55.461240  [ERROR]: rsmi_status_t rsmi_dev_compute_partition_get(uint32_t, char*, uint32_t) | ======= end =======  | Fail  | Device #: 0 | Type: amd::smi::kDevComputePartition | Cause: could not retrieve current compute partition | Returning = RSMI_STATUS_NOT_SUPPORTED: This function is not supported in the current environment. |
2024-04-26 08:52:55.461243  [TRACE]: rsmi_status_t rsmi_dev_unique_id_get(uint32_t, uint64_t*)| ======= start =======
2024-04-26 08:52:55.461255  [INFO]: int amd::smi::Device::openSysfsFileStream(amd::smi::DevInfoTypes, T*, const char*) [with T = std::basic_ifstream<char>] | Successfully opened SYSFS file (/sys/class/drm/card0/device/unique_id) for DevInfoInfoType (amd::smi::kDevUniqueId)
2024-04-26 08:52:55.461262  [INFO]: int amd::smi::Device::readDevInfoStr(amd::smi::DevInfoTypes, std::string*)Successfully read device info string for DevInfoType (amd::smi::kDevUniqueId): 85631fd855c9cea1 |  File stream is closed | [GOOD] No bad bit read, successful read operation | [GOOD] No fail - Successful read operation | [GOOD] No eof error - Successful read operation | [GOOD] read good - Successful read operation
2024-04-26 08:52:55.461269  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | device/node id (cardId) = 0 | card id (cardAdded) = 0 | numMonDevices = 1 | compute partition = UNKNOWN | temp_primary_unique_id = 9611561043751456417 | Num of nodes matching temp_primary_unique_id = 1 | device_uuid (hex/uint) = Hex (MSB): 0x85631fd855c9cea1, Unsigned int: 9611561043751456417, Byte Size: 8 | device_uuid (uint64_t) = 9611561043751456417
2024-04-26 08:52:55.461273  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | REFRESH - primary_unique_id = 9611561043751456417 has 1 known gpu nodes
2024-04-26 08:52:55.461276  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | After finding primary_unique_id = 9611561043751456417 erased 1 nodes
2024-04-26 08:52:55.461292  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card1 is an amdgpu device - TRUE
2024-04-26 08:52:55.461296  [TRACE]: rsmi_status_t rsmi_dev_compute_partition_get(uint32_t, char*, uint32_t)| ======= start =======, dv_ind = 1
2024-04-26 08:52:55.461299  [TRACE]: rsmi_status_t rsmi_dev_unique_id_get(uint32_t, uint64_t*)| ======= start =======
2024-04-26 08:52:55.461302  [TRACE]: void amd::smi::RocmSMI::AddToDeviceList(std::string, uint64_t) | ======= start =======
2024-04-26 08:52:55.461335  [TRACE]: int32_t amd::smi::Monitor::setTempSensorLabelMap() | ======= start =======
2024-04-26 08:52:55.461351  [INFO]: Successfully read SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/temp1_label), returning str = edge
2024-04-26 08:52:55.461355  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp1_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 1 | Data: edge | Returning: 0 |
2024-04-26 08:52:55.461368  [INFO]: Successfully read SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/temp2_label), returning str = junction
2024-04-26 08:52:55.461373  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp2_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 2 | Data: junction | Returning: 0 |
2024-04-26 08:52:55.461390  [INFO]: Successfully read SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/temp3_label), returning str = mem
2024-04-26 08:52:55.461394  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp3_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 3 | Data: mem | Returning: 0 |
2024-04-26 08:52:55.461401  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp4_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 4 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461407  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp5_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 5 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461413  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp6_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 6 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461420  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/temp7_label | Type: MonitorTypes::kMonTempLabel | Sensor id: 7 | Data: mem | Returning: 2 |
2024-04-26 08:52:55.461433  [INFO]: Successfully read SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/in0_label), returning str = vddgfx
2024-04-26 08:52:55.461437  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/in0_label | Type: MonitorTypes::kMonVoltLabel | Sensor id: 0 | Data: vddgfx | Returning: 0 |
2024-04-26 08:52:55.461447  [DEBUG]: uint32_t amd::smi::GetDrmRenderMinor(std::string) | Discovered drmRenderMinor = 129 | For drm_path = /sys/class/drm/card1/device/drm |
2024-04-26 08:52:55.461455  [DEBUG]: void amd::smi::RocmSMI::AddToDeviceList(std::string, uint64_t) | Adding to device list dev_name = card1 | path = /sys/class/drm/card1 | bdfid = 18446744073709551615 | card index = 1 |
2024-04-26 08:52:55.461458  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | Ordered system nodes seen in lookup = {
[node_id = 2; gpu_id = 50916; unique_id = 0; location_id = 2304], }
2024-04-26 08:52:55.461461  [TRACE]: rsmi_status_t rsmi_dev_compute_partition_get(uint32_t, char*, uint32_t)| ======= start =======, dv_ind = 1
2024-04-26 08:52:55.461468  [ERROR]: int amd::smi::Device::openSysfsFileStream(amd::smi::DevInfoTypes, T*, const char*) [with T = std::basic_ifstream<char>] | Issue: File did not exist - SYSFS file (/sys/class/drm/card1/device/current_compute_partition) for DevInfoInfoType (amd::smi::kDevComputePartition), returning 2
2024-04-26 08:52:55.461471  [ERROR]: Could not read device info string for DevInfoType (amd::smi::kDevComputePartition), returning 2
2024-04-26 08:52:55.461474  [ERROR]: rsmi_status_t rsmi_dev_compute_partition_get(uint32_t, char*, uint32_t) | ======= end =======  | Fail  | Device #: 1 | Type: amd::smi::kDevComputePartition | Cause: could not retrieve current compute partition | Returning = RSMI_STATUS_NOT_SUPPORTED: This function is not supported in the current environment. |
2024-04-26 08:52:55.461478  [TRACE]: rsmi_status_t rsmi_dev_unique_id_get(uint32_t, uint64_t*)| ======= start =======
2024-04-26 08:52:55.461484  [ERROR]: int amd::smi::Device::openSysfsFileStream(amd::smi::DevInfoTypes, T*, const char*) [with T = std::basic_ifstream<char>] | Issue: File did not exist - SYSFS file (/sys/class/drm/card1/device/unique_id) for DevInfoInfoType (amd::smi::kDevUniqueId), returning 2
2024-04-26 08:52:55.461490  [ERROR]: Could not read device info string for DevInfoType (amd::smi::kDevUniqueId), returning 2
2024-04-26 08:52:55.461493  [DEBUG]: uint32_t amd::smi::RocmSMI::DiscoverAmdgpuDevices() | rsmi_dev_unique_id_get(cardId, &device_uuid) was not successful, continue..
2024-04-26 08:52:55.461498  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card2 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461502  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card3 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461505  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card4 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461509  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card5 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461513  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card6 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461516  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card7 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461520  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card8 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461524  [DEBUG]: bool amd::smi::isAMDGPU(std::string) | device_path = /sys/class/drm/card9 is an amdgpu device -  FALSE
2024-04-26 08:52:55.461530  [TRACE]: void amd::smi::RocmSMI::Initialize(uint64_t) | [before] device->path() = /sys/class/drm/card0
 | bdfid = 768
 | device->bdfid() = 18446744073709551615
 | (legacy/pcie card) setting device->set_bdfid(bdfid)
2024-04-26 08:52:55.461533  [TRACE]: void amd::smi::RocmSMI::Initialize(uint64_t) | [after] device->path() = /sys/class/drm/card0
 | bdfid = 768
 | device->bdfid() = 768
 | final update: device->bdfid() holds correct device bdf
2024-04-26 08:52:55.461538  [TRACE]: void amd::smi::RocmSMI::Initialize(uint64_t) | [before] device->path() = /sys/class/drm/card1
 | bdfid = 2304
 | device->bdfid() = 18446744073709551615
 | (legacy/pcie card) setting device->set_bdfid(bdfid)
2024-04-26 08:52:55.461541  [TRACE]: void amd::smi::RocmSMI::Initialize(uint64_t) | [after] device->path() = /sys/class/drm/card1
 | bdfid = 2304
 | device->bdfid() = 2304
 | final update: device->bdfid() holds correct device bdf
2024-04-26 08:52:55.461543  [DEBUG]: void amd::smi::RocmSMI::Initialize(uint64_t) Sort index based on BDF.
2024-04-26 08:52:55.462037  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/1/properties | Successfully read node #1 for property_name = gfx_target_version | Data (gfx_target_version) * val = 110000 | return = 0 |
2024-04-26 08:52:55.462047  [DEBUG]: int amd::smi::KFDNode::get_gfx_target_version(uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/1/properties | Successfully read node #1 for gfx_target_version | Data (gfx_target_version) *gfx_target_version = 110000 | return = 0 |
2024-04-26 08:52:55.462130  [DEBUG]: int amd::smi::read_node_properties(uint32_t, std::string, uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/2/properties | Successfully read node #2 for property_name = gfx_target_version | Data (gfx_target_version) * val = 100301 | return = 0 |
2024-04-26 08:52:55.462137  [DEBUG]: int amd::smi::KFDNode::get_gfx_target_version(uint64_t*) | File: /sys/class/kfd/kfd/topology/nodes/2/properties | Successfully read node #2 for gfx_target_version | Data (gfx_target_version) *gfx_target_version = 100301 | return = 0 |
2024-04-26 08:52:55.462143  [INFO]: ====== Gathered system details ============
SYSTEM NAME: Linux
OS DISTRIBUTION: Ubuntu 22.04.4 LTS
NODE NAME: ubuntu-server-220403
RELEASE: 6.5.0-28-generic
VERSION: #29~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  4 14:39:20 UTC 2
MACHINE TYPE: x86_64
DOMAIN: (none)
ENDIANNESS: Little Endian, multi-bit symbols encoded as little endian (LSB first)
ROCM BUILD TYPE: debug
ROCM-SMI-LIB PATH: /opt/rocm-6.0.2/libexec/rocm_smi/../../lib/librocm_smi64.so.6
ROCM-SMI-LIB BUILD DATE: Thu Feb  8 23:54:30 2024
ROCM ENV VARIABLES:
        RSMI_DEBUG_BITFIELD = <undefined>
        RSMI_DEBUG_DRM_ROOT_OVERRIDE = <undefined>
        RSMI_DEBUG_HWMON_ROOT_OVERRIDE = <undefined>
        RSMI_DEBUG_PP_ROOT_OVERRIDE = <undefined>
        RSMI_DEBUG_INFINITE_LOOP = <undefined>
        RSMI_LOGGING = 1
        RSMI_LOGGING (are logs on) = TRUE
        RSMI_DEBUG_ENUM_OVERRIDE = {}
AMD GFX VERSIONS:
        Device[0]: gfx110000
        Device[1]: gfx100301

2024-04-26 08:52:55.462152  [DEBUG]: void amd::smi::RocmSMI::Initialize(uint64_t) | current device paths = Vector = {/sys/class/drm/card0, /sys/class/drm/card1}

2024-04-26 08:52:55.462232  [TRACE]: rsmi_status_t rsmi_dev_vendor_id_get(uint32_t, uint16_t*)| ======= start =======
2024-04-26 08:52:55.462249  [INFO]: int amd::smi::Device::openSysfsFileStream(amd::smi::DevInfoTypes, T*, const char*) [with T = std::basic_ifstream<char>] | Successfully opened SYSFS file (/sys/class/drm/card0/device/vendor) for DevInfoInfoType (amd::smi::kDevVendorID)
2024-04-26 08:52:55.462276  [INFO]: int amd::smi::Device::readDevInfoStr(amd::smi::DevInfoTypes, std::string*)Successfully read device info string for DevInfoType (amd::smi::kDevVendorID): 0x1002 |  File stream is closed | [GOOD] No bad bit read, successful read operation | [GOOD] No fail - Successful read operation | [GOOD] No eof error - Successful read operation | [GOOD] read good - Successful read operation
2024-04-26 08:52:55.462332  [TRACE]: rsmi_status_t rsmi_dev_fan_speed_set(uint32_t, uint32_t, uint64_t)| ======= start =======
2024-04-26 08:52:55.462337  [TRACE]: rsmi_status_t rsmi_dev_fan_speed_max_get(uint32_t, uint32_t, uint64_t*)| ======= start =======
2024-04-26 08:52:55.462383  [INFO]: Successfully read SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/pwm1_max), returning str = 255
2024-04-26 08:52:55.462400  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card0/device/hwmon/hwmon3/pwm1_max | Type: MonitorTypes::kMonMaxFanSpeed | Sensor id: 1 | Data: 255 | Returning: 0 |
2024-04-26 08:52:55.462479  [INFO]: Successfully wrote to SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/pwm1_enable) string = 1
2024-04-26 08:52:55.462526  [INFO]: Successfully wrote to SYSFS file (/sys/class/drm/card0/device/hwmon/hwmon3/pwm1) string = 255
2024-04-26 08:52:55.463207  [TRACE]: rsmi_status_t rsmi_dev_fan_speed_set(uint32_t, uint32_t, uint64_t)| ======= start =======
2024-04-26 08:52:55.463216  [TRACE]: rsmi_status_t rsmi_dev_fan_speed_max_get(uint32_t, uint32_t, uint64_t*)| ======= start =======
2024-04-26 08:52:55.463269  [INFO]: Successfully read SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/pwm1_max), returning str = 255
2024-04-26 08:52:55.463291  [INFO]: int amd::smi::Monitor::readMonitor(amd::smi::MonitorTypes, uint32_t, std::string*) | Success | Read hwmon file: /sys/class/drm/card1/device/hwmon/hwmon4/pwm1_max | Type: MonitorTypes::kMonMaxFanSpeed | Sensor id: 1 | Data: 255 | Returning: 0 |
2024-04-26 08:52:55.858970  [INFO]: Successfully wrote to SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/pwm1_enable) string = 1
2024-04-26 08:52:55.859151  [INFO]: Successfully wrote to SYSFS file (/sys/class/drm/card1/device/hwmon/hwmon4/pwm1) string = 255
Looong01 commented 7 months ago

you can try and trace it to the sysfs file and then edit it directly: sudo strace --trace=file --successful-only /opt/rocm/bin/rocm-smi --setfan 100% 2>&1 | grep sys/class

This will return a bunch of stuff and point you to something like: /sys/class/drm/card1/device/hwmon/hwmon2/pwm1

See if you can set these manually.

I'm expecting that this is a driver issue and amdsmi/rocmsmi can't do anything about it.

P.S. You can also run it with sudo RSMI_LOGGING=1 /opt/rocm/bin/rocm-smi --setfan 100%. it will populate /var/log/rocm_smi_lib/ROCm-SMI-lib.log. =3 will additionally print to console.

I try to reinstall all the Ubuntu system and amdgpu driver and ROCm. It still does NOT work.

$ sudo amd-smi set -f 100% -g 0

            ******WARNING******

            Operating your AMD GPU outside of official AMD specifications or outside of
            factory settings, including but not limited to the conducting of overclocking,
            over-volting or under-volting (including use of this interface software,
            even if such software has been directly or indirectly provided by AMD or otherwise
            affiliated in any way with AMD), may cause damage to your AMD GPU, system components
            and/or result in system failure, as well as cause other problems.
            DAMAGES CAUSED BY USE OF YOUR AMD GPU OUTSIDE OF OFFICIAL AMD SPECIFICATIONS OR
            OUTSIDE OF FACTORY SETTINGS ARE NOT COVERED UNDER ANY AMD PRODUCT WARRANTY AND
            MAY NOT BE COVERED BY YOUR BOARD OR SYSTEM MANUFACTURER'S WARRANTY.
            Please use this utility with caution.

Do you accept these terms? [y/n] y
amdsmi.amdsmi_exception.AmdSmiLibraryException: Error code:
        2 | AMDSMI_STATUS_NOT_SUPPORTED - Feature not supported

The above exception was the direct cause of the following exception:

ValueError: Unable to set fan speed 255 on GPU ID: 0 BDF:0000:03:00.0

and

$ rocm-smi

==================================== ROCm System Management Interface ====================================
============================================== Concise Info ==============================================
Device  [Model : Revision]    Temp    Power  Partitions      SCLK  MCLK   Fan  Perf  PwrCap  VRAM%  GPU%
        Name (20 chars)       (Edge)  (Avg)  (Mem, Compute)
==========================================================================================================
0       [0x7901 : 0xc8]       30.0°C  10.0W  N/A, N/A        0Mhz  96Mhz  0%   auto  327.0W    0%   0%
        0x744c
==========================================================================================================
========================================== End of ROCm SMI Log ===========================================
ppanchad-amd commented 3 months ago

@Looong01 Apologies for the lack of response. Can you please check if your issue still exist with the latest ROCm 6.2? If not, please close the ticket. Thanks!

Looong01 commented 3 months ago

@Looong01 Apologies for the lack of response. Can you please check if your issue still exist with the latest ROCm 6.2? If not, please close the ticket. Thanks!

No, the problem is still here.

$ sudo amd-smi set -g 0 -f 80%

            ******WARNING******

            Operating your AMD GPU outside of official AMD specifications or outside of
            factory settings, including but not limited to the conducting of overclocking,
            over-volting or under-volting (including use of this interface software,
            even if such software has been directly or indirectly provided by AMD or otherwise
            affiliated in any way with AMD), may cause damage to your AMD GPU, system components
            and/or result in system failure, as well as cause other problems.
            DAMAGES CAUSED BY USE OF YOUR AMD GPU OUTSIDE OF OFFICIAL AMD SPECIFICATIONS OR
            OUTSIDE OF FACTORY SETTINGS ARE NOT COVERED UNDER ANY AMD PRODUCT WARRANTY AND
            MAY NOT BE COVERED BY YOUR BOARD OR SYSTEM MANUFACTURER'S WARRANTY.
            Please use this utility with caution.

Do you accept these terms? [y/n] y
amdsmi.amdsmi_exception.AmdSmiLibraryException: Error code:
        2 | AMDSMI_STATUS_NOT_SUPPORTED - Feature not supported

The above exception was the direct cause of the following exception:

ValueError: Unable to set fan speed 204 on GPU ID: 0 BDF:0000:03:00.0
ppanchad-amd commented 3 months ago

@Looong01 Internal ticket is there to get this issue resolved. Thanks!

marifamd commented 2 months ago

@Looong01 Good catch, I'm unable to set my 7900 XTX as well, while my 6900 is working. We'll follow up with the driver team and get this resolved.