search

hakandundar34coding / system-monitoring-center

Multi-featured system monitor
GNU General Public License v3.0
891 stars 81 forks source link

GPU tab provides no stats after GPU memory, encoding, decoding load graphics are added #192

Closed yochananmarqos closed 1 year ago

yochananmarqos commented 1 year ago

I just updated to 2.18.0 and the GPU tab shows nothing for both my integrated Intel graphics and decidated NVIDIA graphics on a hybrid laptop. Note that I have external monitors connected, so the Intel graphics are not used at all.

inxi -Gazy ``` Graphics: Device-1: Intel Alder Lake-P Integrated Graphics vendor: CLEVO/KAPOK driver: i915 v: kernel arch: Gen-12.2 process: Intel 10nm built: 2021-22+ ports: active: DP-2 off: eDP-2 empty: DP-3,DP-4,DP-5 bus-ID: 00:02.0 chip-ID: 8086:46a6 class-ID: 0300 Device-2: NVIDIA GA106M [GeForce RTX 3060 Mobile / Max-Q] vendor: CLEVO/KAPOK driver: nvidia v: 535.54.03 alternate: nouveau,nvidia_drm non-free: 535.xx+ status: current (as of 2023-07) arch: Ampere code: GAxxx process: TSMC n7 (7nm) built: 2020-22 pcie: gen: 2 speed: 5 GT/s lanes: 8 link-max: gen: 4 speed: 16 GT/s lanes: 16 ports: active: none off: DP-1,HDMI-A-1 empty: eDP-1 bus-ID: 01:00.0 chip-ID: 10de:2520 class-ID: 0300 Device-3: Logitech Webcam C270 driver: snd-usb-audio,uvcvideo type: USB rev: 2.0 speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 3-1.1:4 chip-ID: 046d:0825 class-ID: 0102 serial: Device-4: Chicony USB2.0 Camera driver: uvcvideo type: USB rev: 2.0 speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 3-8:5 chip-ID: 04f2:b729 class-ID: fe01 serial: Display: x11 server: X.Org v: 21.1.8 with: Xwayland v: 23.1.2 compositor: gnome-shell v: 44.3 driver: X: loaded: modesetting,nvidia alternate: fbdev,nouveau,nv,vesa dri: iris gpu: i915,nvidia,nvidia-nvswitch display-ID: :1 screens: 1 Screen-1: 0 s-res: 5760x1080 s-dpi: 96 s-size: 1524x286mm (60.00x11.26") s-diag: 1551mm (61.05") Monitor-1: DP-1 note: disabled pos: primary,center model: MSI G27C4 serial: built: 2020 res: 1920x1080 dpi: 82 gamma: 1.2 size: 598x336mm (23.54x13.23") diag: 686mm (27") ratio: 16:9 modes: max: 1920x1080 min: 640x480 Monitor-2: DP-2 mapped: DP-1-2 pos: right model: HP X24ih serial: built: 2021 res: 1920x1080 dpi: 93 gamma: 1.2 size: 527x297mm (20.75x11.69") diag: 605mm (23.8") ratio: 16:9 modes: max: 1920x1080 min: 720x400 Monitor-3: HDMI-A-1 mapped: HDMI-0 note: disabled pos: left model: HP X24ih serial: built: 2021 res: 1920x1080 dpi: 93 gamma: 1.2 size: 527x297mm (20.75x11.69") diag: 605mm (23.8") ratio: 16:9 modes: max: 1920x1080 min: 640x480 Monitor-4: eDP-2 mapped: eDP-1-2 note: disabled model: AU Optronics 0xaf90 built: 2020 res: 1920x1080 dpi: 142 gamma: 1.2 size: 344x193mm (13.54x7.6") diag: 394mm (15.5") ratio: 16:9 modes: 1920x1080 API: OpenGL v: 4.6.0 NVIDIA 535.54.03 renderer: NVIDIA GeForce RTX 3060 Laptop GPU/PCIe/SSE2 direct-render: Yes ``` 
Traceback (most recent call last):
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/MainWindow.py", line 950, in on_row_activated
    Gpu.loop_func()
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Gpu.py", line 609, in loop_func
    gpu_load_memory_frequency_power_dict = Libsysmon.get_gpu_load_memory_frequency_power(gpu_pci_address, device_vendor_id, selected_gpu_number, gpu_list, gpu_device_path_list, gpu_device_sub_path_list)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Libsysmon.py", line 2641, in get_gpu_load_memory_frequency_power
    Thread(target=gpu_load_nvidia_func, daemon=True).start()
    ^^^^^^
NameError: name 'Thread' is not defined
Traceback (most recent call last):
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/MainWindow.py", line 1127, in main_gui_tab_loop
    Gpu.loop_func()
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Gpu.py", line 609, in loop_func
    gpu_load_memory_frequency_power_dict = Libsysmon.get_gpu_load_memory_frequency_power(gpu_pci_address, device_vendor_id, selected_gpu_number, gpu_list, gpu_device_path_list, gpu_device_sub_path_list)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Libsysmon.py", line 2641, in get_gpu_load_memory_frequency_power
    Thread(target=gpu_load_nvidia_func, daemon=True).start()
    ^^^^^^
NameError: name 'Thread' is not defined
hakandundar34coding commented 1 year ago

Can you add threading. at the beginng of the line and try again? But do not remove empty spaces at the beginning of the line. Line 2641 in Libsysmon.py file: Thread(target=gpu_load_nvidia_func, daemon=True).start()

Can you share screenshots for both GPUs if it works?

Note: Integrated Intel GPUs provide limited information in its directories.

yochananmarqos commented 1 year ago

Like this?

threading.Thread(target=gpu_load_nvidia_func, daemon=True).start()

That produces no output, however there are still no stats at all.

hakandundar34coding commented 1 year ago

What about Intel GPU?

yochananmarqos commented 1 year ago

there are still no stats at all.

:wink:

hakandundar34coding commented 1 year ago

Can you share output of this command? nvidia-smi --query-gpu=gpu_name,gpu_bus_id,driver_version,utilization.gpu,utilization.memory,utilization.encoder,utilization.decoder,memory.total,memory.free,memory.used,temperature.gpu,clocks.current.graphics,clocks.max.graphics,clocks.current.memory,clocks.max.memory,power.draw,power.limit --format=csv

yochananmarqos commented 1 year ago 
name, pci.bus_id, driver_version, utilization.gpu [%], utilization.memory [%], utilization.encoder [%], utilization.decoder [%], memory.total [MiB], memory.free [MiB], memory.used [MiB], temperature.gpu, clocks.current.graphics [MHz], clocks.max.graphics [MHz], clocks.current.memory [MHz], clocks.max.memory [MHz], power.draw [W], power.limit [W]
NVIDIA GeForce RTX 3060 Laptop GPU, 00000000:01:00.0, 535.54.03, 0 %, 1 %, 0 %, 0 %, 6144 MiB, 4821 MiB, 1116 MiB, 42, 472 MHz, 2100 MHz, 6000 MHz, 7001 MHz, 22.82 W, [N/A]
hakandundar34coding commented 1 year ago

Can you follow these steps?

The last version may be deleted.

yochananmarqos commented 1 year ago

There are some stats now, however GPU & Memory usage is not populating:

card1 (Intel):

image

card0 (NVIDIA):

image

Output clicking on card0 (NVIDIA):

Traceback (most recent call last):
  File "/home/yochanan/tmp/system-monitoring-center-f3e6fc0c2492f7c7c090ce4158f0d7a72f5eccab/src/MainWindow.py", line 950, in on_row_activated
    Gpu.loop_func()
  File "/home/yochanan/tmp/system-monitoring-center-f3e6fc0c2492f7c7c090ce4158f0d7a72f5eccab/src/Gpu.py", line 253, in loop_func
    gpu_load_memory_frequency_power_dict = Libsysmon.get_gpu_load_memory_frequency_power(gpu_pci_address, device_vendor_id, selected_gpu_number, gpu_list, gpu_device_path_list, gpu_device_sub_path_list)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yochanan/tmp/system-monitoring-center-f3e6fc0c2492f7c7c090ce4158f0d7a72f5eccab/src/Libsysmon.py", line 2641, in get_gpu_load_memory_frequency_power
    Thread(target=gpu_load_nvidia_func, daemon=True).start()
    ^^^^^^
NameError: name 'Thread' is not defined
Traceback (most recent call last):
  File "/home/yochanan/tmp/system-monitoring-center-f3e6fc0c2492f7c7c090ce4158f0d7a72f5eccab/src/MainWindow.py", line 1127, in main_gui_tab_loop
    Gpu.loop_func()
  File "/home/yochanan/tmp/system-monitoring-center-f3e6fc0c2492f7c7c090ce4158f0d7a72f5eccab/src/Gpu.py", line 253, in loop_func
    gpu_load_memory_frequency_power_dict = Libsysmon.get_gpu_load_memory_frequency_power(gpu_pci_address, device_vendor_id, selected_gpu_number, gpu_list, gpu_device_path_list, gpu_device_sub_path_list)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yochanan/tmp/system-monitoring-center-f3e6fc0c2492f7c7c090ce4158f0d7a72f5eccab/src/Libsysmon.py", line 2641, in get_gpu_load_memory_frequency_power
    Thread(target=gpu_load_nvidia_func, daemon=True).start()
    ^^^^^^
NameError: name 'Thread' is not defined
hakandundar34coding commented 1 year ago

Finally, can you add threading. again? Line: Thread(target=gpu_load_nvidia_func, daemon=True).start()

hakandundar34coding commented 1 year ago

Can you test SMC v2.18.1?

yochananmarqos commented 1 year ago

card1 (Intel):

image image

card0 (NVIDIA): image image

Output clicking on card0 (NVIDIA):

Traceback (most recent call last):
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/MainWindow.py", line 950, in on_row_activated
    Gpu.loop_func()
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Gpu.py", line 609, in loop_func
    gpu_load_memory_frequency_power_dict = Libsysmon.get_gpu_load_memory_frequency_power(gpu_pci_address, device_vendor_id, selected_gpu_number, gpu_list, gpu_device_path_list, gpu_device_sub_path_list)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Libsysmon.py", line 2649, in get_gpu_load_memory_frequency_power
    gpu_load_memory_frequency_power_dict = process_gpu_tool_output_nvidia(gpu_pci_address)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/system-monitoring-center/systemmonitoringcenter/Libsysmon.py", line 3117, in process_gpu_tool_output_nvidia
    if gpu_tool_output != "-":
       ^^^^^^^^^^^^^^^
NameError: name 'gpu_tool_output' is not defined
hakandundar34coding commented 1 year ago

Note: Integrated Intel GPUs provide limited information in its directories.

There will not be improvements for integrated Intel GPUs.

This bug is also fixed. Currently, there is no new version. Probably, the error is printed one time. But it does not affect the GPU information shown.

But Max GPU power is not shown. Only current power consumption is shown.

Can you share output of these commands?

nvidia-smi --query-gpu=gpu_name,gpu_bus_id,driver_version,utilization.gpu,utilization.memory,utilization.encoder,utilization.decoder,memory.total,memory.free,memory.used,temperature.gpu,clocks.current.graphics,clocks.max.graphics,clocks.current.memory,clocks.max.memory,power.draw,power.limit,power.limit.enforced --format=csv

nvidia-smi --query-gpu=gpu_name,gpu_bus_id,driver_version,utilization.gpu,utilization.memory,utilization.encoder,utilization.decoder,memory.total,memory.free,memory.used,temperature.gpu,clocks.current.graphics,clocks.max.graphics,clocks.current.memory,clocks.max.memory,power.draw,power.limit,enforced.power.limit --format=csv

yochananmarqos commented 1 year ago 
Field "power.limit.enforced" is not a valid field to query.
name, pci.bus_id, driver_version, utilization.gpu [%], utilization.memory [%], utilization.encoder [%], utilization.decoder [%], memory.total [MiB], memory.free [MiB], memory.used [MiB], temperature.gpu, clocks.current.graphics [MHz], clocks.max.graphics [MHz], clocks.current.memory [MHz], clocks.max.memory [MHz], power.draw [W], power.limit [W], enforced.power.limit [W]
NVIDIA GeForce RTX 3060 Laptop GPU, 00000000:01:00.0, 535.54.03, 3 %, 2 %, 0 %, 0 %, 6144 MiB, 4777 MiB, 1160 MiB, 39, 465 MHz, 2100 MHz, 6000 MHz, 7001 MHz, 21.95 W, [N/A], 80.00 W
hakandundar34coding commented 1 year ago

Can you test the latest source code?

Can you share this information?

yochananmarqos commented 1 year ago

There were no errors.

image

image

hakandundar34coding commented 1 year ago

Can you try again?

yochananmarqos commented 1 year ago

:+1:

image

image

hakandundar34coding commented 1 year ago

What about decoding engine during a video playing?

yochananmarqos commented 1 year ago

The Video Encoder / Decoder graphs do not show anything while nvtop does.

hakandundar34coding commented 1 year ago

Can you share output of the following commands? You can share only elapsed time output parts (last 3-4 lines) of them.

A lot of new parameters added. I do not know if running the command requires long time. I did not learn the result from this thread.

time nvidia-smi --query-gpu=gpu_name,gpu_bus_id,driver_version,utilization.gpu,utilization.memory,utilization.encoder,utilization.decoder,memory.total,memory.free,memory.used,temperature.gpu,clocks.current.graphics,clocks.max.graphics,clocks.current.memory,clocks.max.memory,power.draw,power.limit --format=csv

time nvidia-smi --query-gpu=gpu_name,gpu_bus_id,driver_version,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used,temperature.gpu,clocks.current.graphics,clocks.max.graphics,power.draw --format=csv

hakandundar34coding commented 1 year ago

You can write here if you are bored. Decoding/encoding load bugs may be fixed in another version when another user provides outputs. My GPU is very old and it does not support lots of features because of the driver. GPU tab changes very rare. Because it is one of the hardest part of the application. There were very detailed changes this time. There are a lot of GPUs, models.

yochananmarqos commented 1 year ago 
name, pci.bus_id, driver_version, utilization.gpu [%], utilization.memory [%], utilization.encoder [%], utilization.decoder [%], memory.total [MiB], memory.free [MiB], memory.used [MiB], temperature.gpu, clocks.current.graphics [MHz], clocks.max.graphics [MHz], clocks.current.memory [MHz], clocks.max.memory [MHz], power.draw [W], power.limit [W]
NVIDIA GeForce RTX 3060 Laptop GPU, 00000000:01:00.0, 535.54.03, 8 %, 11 %, 0 %, 0 %, 6144 MiB, 4923 MiB, 1014 MiB, 42, 390 MHz, 2100 MHz, 810 MHz, 7001 MHz, 18.32 W, [N/A]
nvidia-smi  --format=csv  0.00s user 0.01s system 54% cpu 0.013 total
name, pci.bus_id, driver_version, utilization.gpu [%], utilization.memory [%], memory.total [MiB], memory.free [MiB], memory.used [MiB], temperature.gpu, clocks.current.graphics [MHz], clocks.max.graphics [MHz], power.draw [W]
NVIDIA GeForce RTX 3060 Laptop GPU, 00000000:01:00.0, 535.54.03, 1 %, 1 %, 6144 MiB, 4907 MiB, 1030 MiB, 42, 510 MHz, 2100 MHz, 22.97 W
nvidia-smi  --format=csv  0.00s user 0.00s system 48% cpu 0.011 total
hakandundar34coding commented 1 year ago

0.013 total

vs

0.011 total

There is no big change.

What is reported if you use this command during video playback?

nvidia-smi --query-gpu=utilization.encoder,utilization.decoder --format=csv

Does nvtop require root privileges for getting Intel GPU information?

yochananmarqos commented 1 year ago

What is reported if you use this command during video playback?

nvidia-smi --query-gpu=utilization.encoder,utilization.decoder --format=csv

Sorry, turns out I'm not seeing any Encoder / Decoder activity at all. If I remember correctly, MPV needs to be configured manually to use GPU acceleration. I used to have it working in Chromium based browsers, but I've given up on fiddling. Any suggestions?

Does nvtop require root privileges for getting Intel GPU information?

No.

hakandundar34coding commented 1 year ago

There may be problems about nvidia-smi. Video decoding load changes if a video it watched (by using hardware acceleration from web browser, from HDD, etc.). Maybe other GPU is used. I do not know.

Currently there is no code change. nvidia-smi decoding/encoding information is directly shown on the GUI. It looks like it is get as 0.

You can close this issue. Or you can keep it opened if you want to listen different suggestions about this decoding load issue.

Additionally, after some time, a tool like nvtop may be used for getting more detailed GPU information. This depends on popularity of the application.

yochananmarqos commented 1 year ago

As far as this issue is concerned, it was addressed and working well, thank you.

hakandundar34coding commented 1 year ago

You can open new issues for bugs, new features, etc.

After adding another GPU tool for getting GPU information, fixing GPU releated problems may not be that difficult. Currently, Sytem Monitoring Center uses its code for these features.

xrandr dependency is not required for SMC v2.