luckylinux / supermicro-fan-control

Supermicro Fan Control
GNU Affero General Public License v3.0
3 stars 0 forks source link

Run it in TrueNAS #1

Closed slavikme closed 4 months ago

slavikme commented 5 months ago

Any idea how to run it on TrueNAS? TrueNAS doesn't have pip installed and it seems that you can't easily (and not recommended to) install it.

luckylinux commented 5 months ago

Thank you for your Message :smiley:.

I originally developed this for GNU/Linux (as that's my Target Platform).

Other Requirements include:

Since FreeNAS is based FreeBSD, it could be as simple as described here.

I played a bit around with OPNSense and Custom Ports Installation, and I know that the Derivatives of FreeBSD (TrueNAS, OPNSense, ...) tend to have their own Repositories. And mixing several Repositories in FreeBSD is a PITA, neither sane, nor recommended, just a Recipe for disaster.

Try to see which Python Version you have installed:

python3 --version

Or:

python --version

Then try to install the corresponding py3XX-pip Package. If you are lucky, your TrueNAS Repositories have it:

pkg install py3XX-pip

Note that pip might NOT be strictly required. It's just me that prefer to NOT make a mess of the System (and System-Level Python Packages, which can cause System Breakages if Things go wrong), especially since it's NOT allowed by default anymore (on Debian/Ubuntu at least).

luckylinux commented 4 months ago

I updated README and listed (current) Requirements: https://github.com/luckylinux/supermicro-fan-control/commit/52877857a02741b8f1e48c403a1128573b0663b2

luckylinux commented 4 months ago

@slavikme: After a bit more investigation it appears pip Package has been Removed from TrueNAS.

You might be able to follow this Post.

Other alternatives include setting up a FreeBSD Jail and install everything there. But then Permissions to IPMI/System Components might be an issue ...

slavikme commented 4 months ago

@luckylinux Thank you so much for your help and this amazing script. This helps me a lot!

luckylinux commented 4 months ago

@slavikme: You're welcome. Can you please share how you finally solved this (if you did), in case somebody else looks for the same Issue ?

Any adjustments / recommendation / feedback for the Scripts or README ?

slavikme commented 4 months ago

I didn't solved it yet. Once I'll figure this out, I'll share.

This is a very challenging task on TrueNAS/FreeNAS, because of the limitations it postures in its native system.

I think the only solution would be to create a compiled executable instead of running Python's interpreter (which depends on pip to install dependencies).

@luckylinux If you already have a ready environment to test it, can you create such binary and post it to the release section? I think it'll run on my BSD based system.

I think using cython will be the best way to do that. https://stackoverflow.com/a/40057634

P.S If you don't have time for this, lmk and I'll try to configure a suited environment to create this binary.

luckylinux commented 4 months ago

@slavikme: Another Option is to "Port" this to Shell-Only of Course. This Project (supermicro-fan-control) includes both Variable Speed Control as well as Protection.

This Other Project of mine (which is intended to be installed as a Fail-safe / Fallback Mechanism) is currently doing the Protection Only.

But nothing forbids you to add the relevant Functions to it to do the Temperature Regulation. It's just another scope (and instead of being a Generic Tool it becomes a Supermicro/IPMI Specific Tool, at least in that Regards). It's already anyways grabbing the Drives Temperatures, you just need one/two other "IF" Conditions for the Variable Fan Speed. EDIT 1: but of course you also need the dependencies (jq might not be installed/available for Instance).

Python has better libraries and extensibility. The Other Project is BASH Specific, probably not very POSIX Compliant either. Sorry, I'm spoiled, since I almost only used GNU/Linux in my Years and always learned to write scripts for BASH. For OPNSense I had to adapt a bit to tcsh which was a PITA :laughing:.

Binary wise, easiest would be probably for me (or you) to create a LiveUSB Drive with FreeBSD (which version of FreeBSD is tied to your TrueNAS Version ??? Is it FreeBSD 14.1 or FreeBSD 13.3 or something else entirely ?) and create the Binary there. I can do that in a KVM Virtual Machine as well.

But the Question is: even if you have a binary, would you be able to install it / copy over to FreeNAS (/usr/local/bin/XXX) ? And I would assume that also the Configuration Loading needs adaptation (currently the configuration is loaded from /etc/supermicro-fan-control, Program + VENV installed in /opt/supermicro-fan-control).

I guess /etc and maybe /opt are also some Environment Variables which could be accessed though os.getenv or Similar Functions, but do you know the Variable Names ?

Otherwise an if statement based on platform.system() Python Function could separate these two. If you open a Python (Version 3) Prompt and run:

python3
import platform
print(platform.system())

What do you get ?

luckylinux commented 4 months ago

OK I managed using nuitka to get somewhere. cython continued to throw Errors.

Several .so files in one Folder along with the .bin itself.

Several Modifications were also required.

Now I'm looking into if I can make it a single File ...

luckylinux commented 4 months ago

@slavikme : if you want to give it a quick Test ... No warranties of course.

Paths changes from /etc/supermicro-fan-control to /usr/local/etc/supermicro-fan-control based on FreeBSD Directory Structures.

You need of course the other stuff installed (ipmitool, smartctl, etc). supermicro_fan_control.freebsd.amd64.bin.zip

Build Log: 20240629_08h04_FreeBSD_Build_Log.txt

slavikme commented 4 months ago

platform.uname() gives me more information:

uname_result(system='Linux', node='nas2', release='6.6.29-production+truenas', version='#1 SMP PREEMPT_DYNAMIC Wed May 29 15:05:57 UTC 2024', machine='x86_64')
luckylinux commented 4 months ago

platform.uname() gives me more information:

uname_result(system='Linux', node='nas2', release='6.6.29-production+truenas', version='#1 SMP PREEMPT_DYNAMIC Wed May 29 15:05:57 UTC 2024', machine='x86_64')

Linux ????

That seems VERY wrong ... I thought TrueNAS was based on FreeBSD.

They apparently have a version based on Linux as well.

So which one do you have really ? Now I built a binary for FreeBSD ...

slavikme commented 4 months ago

Yes, I am surprised as you are...

I tried to run your executable which didn't succeeded (of corase it was compiled for BSD)

# ./supermicro_fan_control.bin 
zsh: no such file or directory: ./supermicro_fan_control.bin
# ldd supermicro_fan_control.bin
        not a dynamic executable
luckylinux commented 4 months ago

Wait a bit I'll build it for Linux ...

slavikme commented 4 months ago

From Wikipedia:

TrueNAS Core and TrueNAS Enterprise are both FreeBSD based, while TrueNAS Scale is the adaptation of TrueNAS Core based on Debian Gnu/Linux.

Mine is TrueNAS Scale. Yup. GNU/Linux.

Now it explains, why some of the stuff I tried to do before didn't work. Such a rookie mistake 🤦‍♂️

luckylinux commented 4 months ago

Such a Mess ... Like Manufacturers using the same Product Number while changing Components, in every Industry ...

luckylinux commented 4 months ago

Does this work ? Now it's /etc/supermicro-fan-control for Configuration Path since this is Linux :laughing:.

supermicro-fan-control.linux.amd64.bin.zip

slavikme commented 4 months ago

"Permission denied"?!

# ./supermicro-fan-control.linux.amd64.bin 
Error, couldn't launch child (exec): Permission denied
# ls -lh supermicro-fan-control.linux.amd64.bin
-rwxr-xr-x 1 root root 8.3M Jun 29 08:27 supermicro-fan-control.linux.amd64.bin

I wonder what child component refuses to run?

slavikme commented 4 months ago

Maybe it's an ACL issue?

luckylinux commented 4 months ago

In my case as root User it runs fine (as Normal user you don't have Permissions to access /dev/ipmi0 anyways):

[INFO] Loading File /etc/supermicro-fan-control/settings.yaml.default
[DEBUG] Merging Configuration
[DEBUG] Add non-existing Key general in config ({'motherboard': 'default', 'update_interval': 30})
[DEBUG] Add non-existing Key beep in config ({'frequency': 2500, 'duration': 1250, 'delay': 1250, 'repetitions': 5})
[DEBUG] Add non-existing Key cpu in config ({'max_temp': 60, 'min_temp': 50, 'warning_temp': 80, 'shutdown_temp': 95})
[DEBUG] Add non-existing Key drive in config ({'max_temp': 37, 'min_temp': 32, 'warning_temp': 50, 'shutdown_temp': 55})
[DEBUG] Add non-existing Key hdd in config ({'max_temp': 37, 'min_temp': 32, 'warning_temp': 50, 'shutdown_temp': 55})
[DEBUG] Add non-existing Key ssd in config ({'max_temp': 60, 'min_temp': 50, 'warning_temp': 65, 'shutdown_temp': 75})
[DEBUG] Add non-existing Key nvme in config ({'max_temp': 60, 'min_temp': 50, 'warning_temp': 65, 'shutdown_temp': 75})
[DEBUG] Add non-existing Key fan in config ({'max_speed': 100, 'min_speed': 10, 'inc_speed_step': 5, 'dec_speed_step': 1})
[INFO] Loading File /etc/supermicro-fan-control/settings.yaml
[DEBUG] Merging Configuration
[DEBUG] Override Key general in config ({'motherboard': 'default', 'update_interval': 30} -> {'motherboard': 'X10SLM-F', 'update_interval': 30})
[DEBUG] Add non-existing Key general in config ({'motherboard': 'X10SLM-F', 'update_interval': 30})
[INFO] Loading File /etc/supermicro-fan-control/ipmi.d/default.yaml
[DEBUG] Merging Configuration
[DEBUG] Add non-existing Key ipmi in config ({'fan_modes': {'standard': {'registers': ['0x30', '0x45', '0x01', '0x00']}, 'full': {'registers': ['0x30', '0x45', '0x01', '0x01']}, 'optimal': {'registers': ['0x30', '0x45', '0x01', '0x02']}, 'heavy_io': {'registers': ['0x30', '0x45', '0x01', '0x03']}}, 'fan_zones': [{'name': 'Zone 0', 'description': 'CPU Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x00']}, {'name': 'Zone 1', 'description': 'Peripherals Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x01']}]})
[INFO] Loading File /etc/supermicro-fan-control/ipmi.d/X10SLM-F.yaml
[WARNING] File /etc/supermicro-fan-control/ipmi.d/X10SLM-F.yaml is empty
[INFO] Setting Fan Control Mode to Full (Manual)

[INFO] Set Initial Fan Speed to 50%
[INFO] Hex Speed: 0x7f%

^CTraceback (most recent call last):
  File "/tmp/onefile_866254_1719643212_358417/supermicro-fan-control.py", line 444, in <module>
  File "/tmp/onefile_866254_1719643212_358417/supermicro-fan-control.py", line 211, in set_fan_speed
KeyboardInterrupt

Permissions apparmor maybe ? That's the default on Debian, but it's not usually enabled (and if it is, it's usually set to permissive mode).

Try dmesg or cat /var/log/syslog.

It might be that TrueNAS uses SELinux and/or some further Kernel Hardening.

slavikme commented 4 months ago

Nothing in the logs... Wow, this is becoming frustrating.

luckylinux commented 4 months ago

@slavikme: actually it could also be something else: https://github.com/Nuitka/Nuitka/issues/2246 https://github.com/Nuitka/Nuitka/issues/2142

Try to check with filesystem is mounted with "noexec":

mount -l | grep -i noexec
luckylinux commented 4 months ago

Let me try to see if building with --onefile-tempdir-spec=ONEFILE_TEMPDIR_SPEC will work.

slavikme commented 4 months ago

I run it from /root

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=26400928k,mode=755,inode64)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=102400k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,inode64)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysusers.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
slavikme commented 4 months ago

You think it uses /tmp?

luckylinux commented 4 months ago

It does ! Look at the last Lines in my Log !

I originally foresaw to use /opt/supermicro-fan-control and then bin , venv, etc. So I can build that it will use /opt/supermicro-fan-control/tmp.

But it's going to be static then. I cannot do 1000 builds for everybody with a different Path !

Either that, or you need to change the mount options for /tmp.

I don't see many Options.

luckylinux commented 4 months ago

@slavikme: Try this

supermicro-fan-control.linux.separatetmp.amd64.bin.zip

You will need to create /opt/supermicro-fan-control/tmp at the very least (plus the configuration files to /etc/supermicro-fan-control/...).

slavikme commented 4 months ago

I've changed the evn var TMPDIR to something else, and now it gives me another error:

./supermicro-fan-control.linux.amd64.bin: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.38' not found (required by ./supermicro-fan-control.linux.amd64.bin)
./supermicro-fan-control.linux.amd64.bin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by ./supermicro-fan-control.linux.amd64.bin)
./supermicro-fan-control.linux.amd64.bin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /root/downloads/supermicro-fan-control/build/linux/onefile_2251659_1719643971_280076/libexpat.so.1)
luckylinux commented 4 months ago

I build this on Ubuntu so I have a newer GLIB Version. I'll now need to rebuild on Debian Stable. What a Mess this TrueNAS !

Wait a bit ... I'll build on Debian Bookworm now.

slavikme commented 4 months ago

Don't worry about it, you've done more than enough. Maybe this is not the right path to go.

I need to go now and try to figure this out later today.

Thank you so much for you assistance!!! You are the best! ❤️

luckylinux commented 4 months ago

I'm building now for Debian Bookworm. So you want to manually override using TMPDIR ? I think that's the most flexible for everybody.

slavikme commented 4 months ago

For now manually override TMPDIR is best solution, because for me /opt is a Read-only file system.

luckylinux commented 4 months ago

That's one of the reasons I never like to install Appliances. They are a PITA to customize. Only Place where I make an Exception is OPNSense.

luckylinux commented 4 months ago

It's currently Building for Debian Bookworm 12 AMD64. It will take a few more Minutes.

slavikme commented 4 months ago

Yeah, I totally understand you.

slavikme commented 4 months ago

This is my version

# ldd --version
ldd (Debian GLIBC 2.36-9+deb12u4) 2.36
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
luckylinux commented 4 months ago

Mine should be close enough:

# ldd --version
ldd (Debian GLIBC 2.36-9+deb12u7) 2.36
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

@slavikme: Try this

supermicro-fan-control.linux.debian12.amd64.bin.zip

slavikme commented 4 months ago

Hallelujah!!!

Thank you so much!

[INFO] Loading File /etc/supermicro-fan-control/settings.yaml.default
[DEBUG] Merging Configuration
[DEBUG] Add non-existing Key general in config ({'motherboard': 'default', 'update_interval': 30})
[DEBUG] Add non-existing Key beep in config ({'frequency': 2500, 'duration': 1250, 'delay': 1250, 'repetitions': 5})
[DEBUG] Add non-existing Key cpu in config ({'max_temp': 60, 'min_temp': 50, 'warning_temp': 80, 'shutdown_temp': 95})
[DEBUG] Add non-existing Key drive in config ({'max_temp': 37, 'min_temp': 32, 'warning_temp': 50, 'shutdown_temp': 55})
[DEBUG] Add non-existing Key hdd in config ({'max_temp': 37, 'min_temp': 32, 'warning_temp': 50, 'shutdown_temp': 55})
[DEBUG] Add non-existing Key ssd in config ({'max_temp': 60, 'min_temp': 50, 'warning_temp': 65, 'shutdown_temp': 75})
[DEBUG] Add non-existing Key nvme in config ({'max_temp': 60, 'min_temp': 50, 'warning_temp': 65, 'shutdown_temp': 75})
[DEBUG] Add non-existing Key fan in config ({'max_speed': 100, 'min_speed': 10, 'inc_speed_step': 5, 'dec_speed_step': 1})
[INFO] Loading File /etc/supermicro-fan-control/settings.yaml
[DEBUG] Merging Configuration
[DEBUG] Override Key general in config ({'motherboard': 'default', 'update_interval': 30} -> {'motherboard': 'default', 'update_interval': 30})
[DEBUG] Add non-existing Key general in config ({'motherboard': 'default', 'update_interval': 30})
[INFO] Loading File /etc/supermicro-fan-control/ipmi.d/default.yaml
[DEBUG] Merging Configuration
[DEBUG] Add non-existing Key ipmi in config ({'fan_modes': {'standard': {'registers': ['0x30', '0x45', '0x01', '0x00']}, 'full': {'registers': ['0x30', '0x45', '0x01', '0x01']}, 'optimal': {'registers': ['0x30', '0x45', '0x01', '0x02']}, 'heavy_io': {'registers': ['0x30', '0x45', '0x01', '0x03']}}, 'fan_zones': [{'name': 'Zone 0', 'description': 'CPU Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x00']}, {'name': 'Zone 1', 'description': 'Peripherals Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x01']}]})
[INFO] Loading File /etc/supermicro-fan-control/ipmi.d/default.yaml
[DEBUG] Merging Configuration
[DEBUG] Override Key ipmi in config ({'fan_modes': {'standard': {'registers': ['0x30', '0x45', '0x01', '0x00']}, 'full': {'registers': ['0x30', '0x45', '0x01', '0x01']}, 'optimal': {'registers': ['0x30', '0x45', '0x01', '0x02']}, 'heavy_io': {'registers': ['0x30', '0x45', '0x01', '0x03']}}, 'fan_zones': [{'name': 'Zone 0', 'description': 'CPU Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x00']}, {'name': 'Zone 1', 'description': 'Peripherals Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x01']}]} -> {'fan_modes': {'standard': {'registers': ['0x30', '0x45', '0x01', '0x00']}, 'full': {'registers': ['0x30', '0x45', '0x01', '0x01']}, 'optimal': {'registers': ['0x30', '0x45', '0x01', '0x02']}, 'heavy_io': {'registers': ['0x30', '0x45', '0x01', '0x03']}}, 'fan_zones': [{'name': 'Zone 0', 'description': 'CPU Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x00']}, {'name': 'Zone 1', 'description': 'Peripherals Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x01']}]})
[DEBUG] Add non-existing Key ipmi in config ({'fan_modes': {'standard': {'registers': ['0x30', '0x45', '0x01', '0x00']}, 'full': {'registers': ['0x30', '0x45', '0x01', '0x01']}, 'optimal': {'registers': ['0x30', '0x45', '0x01', '0x02']}, 'heavy_io': {'registers': ['0x30', '0x45', '0x01', '0x03']}}, 'fan_zones': [{'name': 'Zone 0', 'description': 'CPU Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x00']}, {'name': 'Zone 1', 'description': 'Peripherals Zone', 'registers': ['0x30', '0x70', '0x66', '0x01', '0x01']}]})
[INFO] Setting Fan Control Mode to Full (Manual)
IANA PEN registry open failed: No such file or directory

[INFO] Set Initial Fan Speed to 50%
[INFO] Hex Speed: 0x7f%
IANA PEN registry open failed: No such file or directory

IANA PEN registry open failed: No such file or directory

[INFO] Fan speed adjusted to 50%
[INFO] Fan speed adjusted to 50% - Hex: 0x7f
IANA PEN registry open failed: No such file or directory
[INFO] Current CPU Temperature: 50°C
[INFO] No HDD Detected
[INFO] SSD Drive /dev/disk/by-id/ata-SATADOM-SL_3ME3_V2_BCA11*** has Temperature = 30.0°C
[INFO] Maximum SSD Temperature: 30.0°C
[INFO] NVME Drive /dev/disk/by-id/nvme-Samsung_SSD_960_EVO_1TB_S3X3N***_1 has Temperature = 40.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.000000000000001000080d0*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.0025385c*** has Temperature = 41.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_489F***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_489F*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.000000000000001000080d0*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.000000000000001000080d0*** has Temperature = 37.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.000000000000001000080d0*** has Temperature = 37.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 37.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-INTEL_SSDPE2NV153T8_BTLL93***_1 has Temperature = 44.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-INTEL_SSDPE2NV153T8_BTLL93***_1 has Temperature = 43.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-Samsung_SSD_960_EVO_1TB_S3X3***_1 has Temperature = 37.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.000000000000001000080d0*** has Temperature = 34.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.0025385c*** has Temperature = 37.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-eui.000000000000001000080d0*** has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-Samsung_SSD_960_EVO_1TB_S3X3***_1 has Temperature = 38.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-Samsung_SSD_960_EVO_1TB_S3X3*** has Temperature = 38.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF***_1 has Temperature = 35.85°C
[INFO] NVME Drive /dev/disk/by-id/nvme-KXG50PNV2T04_TOSHIBA_38KF*** has Temperature = 35.85°C
[INFO] Maximum NVME Temperature: 44.85°C
[DEBUG] Skipping Fan Speed Update for CPU Controller since CPU Temperature = 50°C is within Histeresis Range = [50°C ... 60°C]
[DEBUG] HDD Temperature = 0°C is lower than the Warning Setting = 50°C. No Action required.
[DEBUG] Decreasing Fan Speed since HDD Temperature = 0°C is lower than the Minimum Setting = 32°C
[DEBUG] New Fan Speed based on HDD Controller Temperature = 49%
[DEBUG] SSD Temperature = 30.0°C is lower than the Warning Setting = 65°C. No Action required.
[DEBUG] Decreasing Fan Speed since SSD Temperature = 30.0°C is lower than the Minimum Setting = 50°C
[DEBUG] New Fan Speed based on SSD Controller Temperature = 49%
[DEBUG] NVME Temperature = 44.85°C is lower than the Warning Setting = 65°C. No Action required.
[DEBUG] Decreasing Fan Speed since NVME Temperature = 44.85°C is lower than the Minimum Setting = 50°C
[DEBUG] New Fan Speed based on NVME Controller Temperature = 49%
[DEBUG] No Fan Speed Update required. Keeping Fan Speed to 50%
slavikme commented 4 months ago

Now you can post all these binaries in the release section here, so others could enjoy this amazing creation!

luckylinux commented 4 months ago

Keep in mind it's not even tagged. It's like pre-alpha or something what I gave you :rofl:.

I don't know how to setup CI/CD Pipelines in Github. I have some Machines I can use, but how to configure automatic builds is another Story ...

For the Warning about IANA PEN it's a known Debian Issue which can be fixed like this: https://www.truenas.com/community/threads/truenas-core-13-u6-1-scale-23-10-2-migration-feedback.117183/#post-813499

luckylinux commented 4 months ago

Please check that it also does what it's supposed to do, e.g. with ipmitool sensor and look at the fan speeds.

Different Motherboards can have different IPMI Registers (that's why there is a default profile, but also the possibility to configure custom Motherboard Profiles).

luckylinux commented 4 months ago

And i suggest you also install the Fallback Protection System.

slavikme commented 4 months ago

By the way, this is my metal https://www.supermicro.com/en/products/system/2U/2028/SSG-2028R-DN2R48L.cfm

slavikme commented 4 months ago

Currently I have only one fan (removed the others to reduce noise), that spins around 13300 rpm. And I needed this script to put back other fans and to be controlled automatically. Currently, the system fails to do that correctly for some reason.

Anyway, it doesn't seems like it changed the only fan I have. Will try to put back all others and see the result.

# ipmitool sensor | grep FAN
MB-FAN1          | na         |            | na    | na        | na        | na        | na        | na        | na        
MB-FAN2          | na         |            | na    | na        | na        | na        | na        | na        | na        
BPN-FAN1         | na         |            | na    | na        | na        | na        | na        | na        | na        
BPN-FAN2         | 13300.000  | RPM        | nc    | 500.000   | 400.000   | 300.000   | 13300.000 | 13400.000 | 13500.000 
BPN-FAN3         | na         |            | na    | na        | na        | na        | na        | na        | na        
BPN-FAN4         | na         |            | na    | na        | na        | na        | na        | na        | na        
BPN-FAN5         | na         |            | na    | na        | na        | na        | na        | na        | na
luckylinux commented 4 months ago

I just did the Release: https://github.com/luckylinux/supermicro-fan-control/releases/tag/v0.1.0

luckylinux commented 4 months ago

@slavikme: As I said, you probably need to submit a Motherboard specific Profile that I can integrate into the Code into etc/supermicro-fan-control/ipmi.d/<motherboard>.yml. And configure the Motherboard in etc/supermicro-fan-control/settings.yml.

Supermicro does different things for X10, X11, etc Motherboards somewhat.

You need to dig up the Register Numbers. See the References here, especially the ServeTheHome Forum ones.

I cannot test that Part for you. If Supermicro does things differently on every board, it's impossible. Once you figure it out, please submit a PR and I'll include the Profile into the main Code of Course.

luckylinux commented 4 months ago

The other Option is that you already had "Fan Mode" to "Full". Try to toggle that to first "Standard", then re-run the Script (which should set the "Fan Mode" to "Full" at Startup).

luckylinux commented 4 months ago

For instance here for X9 Motherboard the Registers are very different: 0x30 0x91 0x5A 0x03 0x01 from what I currently use as "default" Profile which I tested on X10SLM-F/X10SLL-F (0x30 0x70 0x66 0x01 0x00)

luckylinux commented 4 months ago

@slavikme:

On my side, I probably will need to implement in the code some ipmitool return code check (currently none is implemented) to detect if there has been any warning.

What does ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x7f (50% speed = 0x7f) say ? Any Error ?

luckylinux commented 4 months ago

@slavikme: I changed the code a bit. Not a lot of Functional Changes, mainly more clarity into logging and also Logging the Measured Fan Speed now.

If you can help me setup a CI/CD Pipeline on GitHub (I have no Experience with it unfortunately) that does the Following in a Debian Bookworm 12 AMD64 VM / Container:

git clone https://github.com/luckylinux/supermicro-fan-control.git
cd supermicro-fan-control
./build.sh

Then the whole process could get automated :+1:.

Otherwise I guess this is something that yourself could also do, if you need the latest bleeding-edge.

slavikme commented 4 months ago

@luckylinux Thank you for all the info, this really helps.

Unfortunately, I was very busy recently and didn't had time to make additional tests/checks. I'm not even sure if I'll find more time this or next week.

Sure, I'll be happy to help you setup a CI/CD pipeline as soon as I free.

By the way, the ipmitools raw does successfully executed without any errors, despite the fact that it doesn't change the fan speed at all.

I will try to find more codes using the links you've provided, maybe I'll discover more findings on fan controls of my motherboard.