handling of multiple fans on Lenovo P50

[sassmann]

The P50 has 2 fans, lm-sensors shows:

thinkpad-isa-0000
Adapter: ISA adapter
fan1:        2316 RPM
fan2:        2332 RPM

in sysfs there is

# ls -al /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon3/
total 0
drwxr-xr-x 3 root root    0 Aug 14 13:39 .
drwxr-xr-x 3 root root    0 Aug 14 10:01 ..
lrwxrwxrwx 1 root root    0 Aug 14 10:01 device -> ../../../thinkpad_hwmon
-r--r--r-- 1 root root 4096 Aug 14 13:39 fan1_input
-r--r--r-- 1 root root 4096 Aug 14 10:01 fan2_input
-r--r--r-- 1 root root 4096 Aug 14 10:01 name
drwxr-xr-x 2 root root    0 Aug 14 13:39 power
-rw-r--r-- 1 root root 4096 Aug 14 13:39 pwm1
-rw-r--r-- 1 root root 4096 Aug 14 13:39 pwm1_enable
lrwxrwxrwx 1 root root    0 Aug 14 10:01 subsystem -> ../../../../../class/hwmon
-rw-r--r-- 1 root root 4096 Aug 14 10:01 uevent

It seems that thinkfan is only able to control fan1, fan2 keeps constant ~2300 RPM. thinkfan.conf used

tp_fan /proc/acpi/ibm/fan
hwmon /sys/devices/platform/coretemp.0/hwmon/hwmon4/temp1_input
(0,     0,      65)
(1,     55,     82)
(2,     65,     88)
(6,     82,     92)
(7,     70,     32767)

Is there a way to control both fans?

[lhofhansl]

@sassmann So TPACPI_Q_LNV('N', '1', TPACPI_FAN_2FAN) does not cover it? Since 'N', '1' is a prefix of 'N', '1', 'E', I thought that should work.

Edit: Looked at the code again. TPACPI_Q_LNV3('N', '1', 'E', TPACPI_FAN_2FAN) should not be needed, since the BIOS id is already matched by TPACPI_Q_LNV('N', '1', TPACPI_FAN_2FAN).

Are you 100% you needed this? Edit2: Because that would be a bug in the patch that I'd need to correct quickly. :)

[ratcashdev]

asked the manjaro team to add this patch to the list of patches in their kernels: https://gitlab.manjaro.org/packages/core/linux56/-/issues/5

[sassmann]

@lhofhansl ah I didn't notice the TPACPI_Q_LNV('N', '1', TPACPI_FAN_2FAN), so I will do some more testing. If I don't get to it before the patch is resubmitted just proceed without it. We can always add it later on if necessary.

[LukeFernandes]

@tobomobo @LukeFernandes I do not know any specifics about Manjaro. :(

If you build the modules the first time, perhaps you'll need to run make config first. (You can just hit enter through all the defaults - doesn't matter for the module) After that make oldconfig as in the script is all you need.

Does that help?

Re: 4.19... Lemme try (or have you managed to apply the patch?)

Thanks - especially for the tip to press enter throughout (never built a kernel before - maybe that could go in the readme?).

I'm still having the issues though... despite make config, none of my existing Manjaro kernels have .config and Module.symvers. I'm not sure how make config would work here anyway since make would be working in the 5.6.4 kernel folder the .sh script downloads, not the existing kernel.

[sassmann]

@lhofhansl fan2 and the "secondary fan support enabled" message only show up with TPACPI_Q_LNV3('N', '1', 'E', TPACPI_FAN_2FAN)

[lhofhansl]

Thanks @sassmann ! Meh. :(

[civic9]

TPACPI_Q_LNV3 is needed now for models with xxxyTkkW bios/EC id pattern, TPACPI_Q_LNV can't be used. look at tpacpi_parse_fw_id.

[lhofhansl]

Thanks @sassmann @civic9. Posted a new patch to the kernel list just now. That is even a bit safer since it retains the old behavior and only adds 2nd fan control for the listed new H/W. I took the H/W ids from your patch @civic9 (and you are credited in the patch).

Edit: spelling

[lhofhansl]

@LukeFernandes You need to have the kernel source for the 4.19 kernel you're running. (or checkout the kernel git repo to and v4.19.x tag - that's what I do.)

[ratcashdev]

patch is now merged to the manjaro kernel package: https://gitlab.manjaro.org/packages/core/linux56/-/issues/5 If you do any further changes to the patch, please consider notifying the manjaro team.

[LukeFernandes]

@LukeFernandes You need to have the kernel source for the 4.19 kernel you're running. (or checkout the kernel git repo to and v4.19.x tag - that's what I do.)

Thanks - sorry to be an absolute pain but I'm following this thread (https://forum.manjaro.org/t/download-kernel-source/104920/2) and I'm downloading the source, but when I run make config I don't know where to find .config and Module.symvers - so that I can put them in the build folders (which the script expects: cp /usr/lib/modules/$(uname -r)/build/.config ./ cp /usr/lib/modules/$(uname -r)/build/Module.symvers ./).

I don't understand what these two folders are so I don't understand their relationship to the kernel source.

[lhofhansl]

@ratcashdev I posted a new patch to the kernel list (V4). I don't know how to reach the manjaro folks, but it would really be best if they followed the mailing lists.

See: https://www.spinics.net/lists/platform-driver-x86/msg21463.html

[lhofhansl]

@LukeFernandes I understand. Building the Kernel can be daunting. I do not know what manjaro has changed (if anything) in the build process, though.

I cloned the pristine Kernel source from GitHub and built from there.

[vmatare]

@lhofhansl I've just merged the multifan branch in the the master and took a look a look at your patch. From the patch itself I can't tell how control for the second fan is exposed to userspace. Does it actually create a pwm2 control file under /sys/class/hwmon or is it only accessible through /proc/acpi/ibm/fan?

[lhofhansl]

In both cases it's represented as a single fan. So /sys/.../pwm1 or /proc/acpi/ibm/fan always controls both fans. No multifan userspace support is needed, in simply looks like a single fan, I've been using it this way since April.

There are two reasons for that: (1) there was no release userspace tool until now :) and (2) the multifan kernel patch was not coded well - it maintained internal state that needed to be tracked and graceful fallback in case of problems was unclear.

I'd like to get the current treat-both-fans-as-one patch in, then improve to optionally control both fans independently (although for the Thinkpads I see no strong reasons to do that).

[peter-stoll]

@lhofhansl

@peter-stoll The P1/X1E gen1 are supported via this existing quirk TPACPI_Q_LNV('N', '1', TPACPI_FAN_2FAN), the patch extends their functionality to fan control (instead of just querying)

Thanks for that. I use the patch now for about 2 weeks on my P1 Gen 2 and can confirm that it works without any problem.

[lhofhansl]

Likewise I've been using this for past 5 weeks, and it has been working fine. (X1E Gen2)

[ratcashdev]

also using it for X1 Gen 1 for several weeks now. However, if anyone cares to post a config, I'd appreciate it. With the config below I have, the area around F2-F4 keys are untouchably hot when, for example, compiling code for a longer period of time and all cores are running at 100%. Temps reach 95+ (which is expected), but the surface of the laptop should not be that hot, I guess, or should it?

sensors:
  - hwmon: /sys/devices/platform/thinkpad_hwmon/hwmon
    indices: [1]

fans:
  - hwmon: /sys/devices/platform/thinkpad_hwmon/hwmon
    indices: [1]

#
# Simple level config
# meaning: [FANSPEED, LOWER_LIMIT, UPPER_LIMIT]
#
levels:
  - [0, 0, 55]
  - [32, 53, 70]
  - [64, 68, 80]
  - [96, 78, 85]
  - [128, 83, 90]
  - [160, 88, 92]
  - [255, 90, 32767]

[vmatare]

@ratcashdev

but the surface of the laptop should not be that hot, I guess.

Well, it's a complaint heard from many older MacBook users as well. Thermodynamically, if the laptop's surface is hot that means there is good thermal conductivity from the heat-producing components to the outer casing. And that is absolutely a good thing. You precisely want to get the heat away from the sensitive electronics and distribute it as evenly as possible across as large a surface as possible. That's precisely what a heatspreader does. And using the entire laptop case as a heatspreader is definitely good design, although it is apparently somewhat irritating to many users ;-)

[lhofhansl]

My X1E2 stays relatively cool. Here's my config:

tp_fan /proc/acpi/ibm/fan

(0,     0,      54)
(1,     42,     60)
(2,     50,     62)
(3,     52,     64)
(4,     56,     66)
(5,     59,     68)
(7,     63,     32767)

Important. Turbo mode is off! (Turbo mode is a non-sustainable power mode and really heats up the machine). echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo

Edit: I can't stress the turbo mode part enough. It should really be off by default - but it isn't.

With that even when compiling with all cores (and HTs) active the fans comfortably keep the temperature below 60. (It also means it won't go above 2.3GHz.)

[lhofhansl]

Also. I found that the memory card reader prevent the CPU package from going into lower c-states. You can:

1. Disable in the bios (disable IO access in the security tab in the bios)
2. Ignore low LTR for the the Southbridge: echo 0 > /sys/kernel/debug/pmc_core/ltr_ignore
3. Or issue:

   $ sudo setpci -s 53:00.0 800.l=1
   $ sudo setpci -s 53:00.0 5c.l=08010c01
   $ sudo setpci -s 53:00.0 800.l=0

   I'd be careful with the last one, someone from the Kernel list helped me out with that.

You can see your powerstates with powertop. In the "Idle Stats" tab you should see the package entering pc10. This all applies to the X1E gen2 (and probably P1 gen2) only

[ratcashdev]

Thank you for the config. A bit louder, but definitely better temps. P1 Gen 2: I could not reach pc10 neither with the BIOS (setting MC IO Access enable to FALSE) nor with the kernel method (I am getting setpci: Warning: No devices selected for "800.l=1". for the first line so I did not continue). I'd love learn more about, but I don't want to hijack this thread.

Another thing is that even I do have the kernel patch, the fans report different RPMS:

cat /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan?_input 
2467
2180

this ~300 RPM difference seems to be constant between them.

[lhofhansl]

Yes. I see the same. They seem to have different rpms at the whatever a level setting is. My fans hardly turn out. You might want to check your other setting, perhaps install TLP.

@ratcashdev if you want you can post the output of

cat /sys/kernel/debug/pmc_core/ltr_show

here. That will show which devices is responsible for preventing the sleeping.

Or you can check of there's a process that wakes up very frequently (you can see that powertop).

@vmatare Sorry for hijacking this issue...

[ratcashdev]

@lhofhansl I sent you an email. Let's leave this thread for fan stuff. I do have TLP installed and running, actually.

[buttertoad]

@lhofhansl Just wanted to clarify a couple things. When you talk about the fans moving together do you mean that they always come on together, run at the same RPM, or both? I have a P73 (N2N) and in firmware mode they act separately. The left fan seems to be for the CPU and rest of the laptop while the right fan only seems to come on when the GPU is hot.

I installed civic9's patch in the thinkpad_acpi module and I can see both fans now and I use a modified thinkfan-control-gui to control the fans. I send the same PWM value to both fans right now because they do share a heatpipe and it makes a big difference overall, but they do not run at the same RPMs. The GPU fan runs faster then the CPU fan.

So, when you talk about running the fans together do you mean sending the same PWM value to both and seeing them as two fans or would we only see one fan readout. If it's only one how would we know which fan we are seeing?

Thanks to everyone for their work on this. I know I for one really appreciate it.

[medwatt]

I am trying to control my fan using Fedora for my P50. Like other users, only 1 fan is detected. I installed the thinkfan module and the first fan is now getting controlled. However, the second fan is being a nuisance sometimes. For example, during a restart, it kicks in and never seems to want to stop.

thinkpad-isa-0000 Adapter: ISA adapter fan1: 0 RPM temp1: +44.0°C
temp2: +46.0°C
temp3: +0.0°C
temp4: +0.0°C
temp5: +0.0°C
temp6: +0.0°C
temp7: +0.0°C
temp8: +0.0°C

nvme-pci-3e00 Adapter: PCI adapter Composite: +67.9°C (low = -273.1°C, high = +69.8°C) (crit = +79.8°C)

I've noticed that during restart, nvme-pci-3e00 goes as high as 69.8°C and this causes the second fan to stay on. For some reason, this temperature doesn't change at all unless I put the system to sleep and immediately turn it back on. Then the temperature "magically" goes to room temperature and the second fan turns off. Seems that there might be a high temperature bug. Any idea how to fix this ?

[sassmann]

@medwatt try kernel 5.8-rc1 or later. Thinkfan settings should apply to both fans now.

[medwatt]

@medwatt try kernel 5.8-rc1 or later. Thinkfan settings should apply to both fans now.

When was this released?

[sassmann]

Upstream released it yesterday, but it won't hit Fedora for quite a while. If you want to test it early you will have to rebuild your fedora kernel with the following patch included. 14232c6e788c platform/x86: thinkpad_acpi: Add support for dual fan control

[medwatt]

Upstream released it yesterday, but it won't hit Fedora for quite a while. If you want to test it early you will have to rebuild your fedora kernel with the following patch included. 14232c6e788c platform/x86: thinkpad_acpi: Add support for dual fan control

I have been following this thread and was waiting for version 5.8. Just didn't know it would be out this soon. Do you know if the latest kernel is available on Manjaro ?

[ratcashdev]

@medwatt manjaro has the patches since version 5.4 applied already. https://gitlab.manjaro.org/packages/core/linux56/-/issues/5 With 5.8 the support is directly in mainline kernel.

[medwatt]

@medwatt manjaro has the patches since version 5.4 applied already. https://gitlab.manjaro.org/packages/core/linux56/-/issues/5 With 5.8 the support is directly in mainline kernel.

I just installed Manjaro. I'm on kernel version 5.6.14. I still get only fan1.

[ratcashdev]

strange:

$ uname -a
Linux zzz-manjaro 5.7.0-3-MANJARO #1 SMP PREEMPT Wed Jun 3 03:15:33 UTC 2020 x86_64 GNU/Linux
$ cat /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan?_input 
3271
2964
$ ls /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan?_input 
/sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan1_input  /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan2_input

Edit: I just realized, that manjaro has a slightly older version of the patch. May not have the quirk settings for your laptop, yet.

[medwatt]

strange:

$ uname -a
Linux zzz-manjaro 5.7.0-3-MANJARO #1 SMP PREEMPT Wed Jun 3 03:15:33 UTC 2020 x86_64 GNU/Linux
$ cat /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan?_input 
3271
2964
$ ls /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan?_input 
/sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan1_input  /sys/devices/platform/thinkpad_hwmon/hwmon/hwmon7/fan2_input

Edit: I just realized, that manjaro has a slightly older version of the patch. May not have the quirk settings for your laptop, yet.

I just installed the same kernel version as yours and still only see fan1.

[ratcashdev]

@medwatt in that case you'll have to build your own acpi module with the latest version of the patch or try 5.8-rc1

[medwatt]

I just noticed that after installing Thinkfan on Manjaro on my P50, everything works as expected. For some reason, as soon as I reboot again, I get a message saying "failed to start fan control program". If I uninstall everything and do install again, it works the first time and stops working after the second reboot. Any ideas what might be causing this ?

[ratcashdev]

@medwatt my guess would be that the index in the hwmon* changes and thus your thinkfan config becomes invalid. Check the wiki for the yaml config.

[medwatt]

@medwatt my guess would be that the index in the hwmon* changes and thus your thinkfan config becomes invalid. Check the wiki for the yaml config.

I guess this is too complicated for me. I'm trying to get into Linux and already about to give up. It requires too much fidgeting to get it to work.

[voidworker]

Does 5.8-rc1 handles both fans as one? :( Now we definitely need a (probably unstable) patch to get separate control back...

[ratcashdev]

@medwatt I share your frustration. Linux is still not the zero-config OS. It is much better though than it was a couple of years back. You don't have to do everything at once, and I am sure over time it will get even better. Anyway, if you're willing to give it one more try, get the config I posted here: https://github.com/vmatare/thinkfan/issues/58#issuecomment-642446558 save it as /etc/thinkfan.conf, restart, and see what happens.

[medwatt]

@medwatt I share you frustration. Linux is still not the zero-config OS. It is much better though than it was a couple of years back. You don't have to do everything at once, and I sure over time it will get even better. Anyway, if you're willing to give it one more try, get the config I posted here: #58 (comment) save it as /etc/thinkfan.conf, restart, and see what happens.

I know nobody cares whether I stick to Linux or not, but it's just too frustrating sometimes that you feel like giving up. You install a distro and everything feels perfect except that the damn fan doesn't shut off. You try to solve it and you get overwhelmed with the solutions. I've barely used Linux for a few days and I'm expected to build my own custom kernel to fix the fan speed problem. I was on KDE and noticed a animations were very janky when using the Nvidia GPU. I tried all methods online and couldn't. Anyway, let me try your script. I hope it works.

[medwatt]

@medwatt I share you frustration. Linux is still not the zero-config OS. It is much better though than it was a couple of years back. You don't have to do everything at once, and I sure over time it will get even better. Anyway, if you're willing to give it one more try, get the config I posted here: #58 (comment) save it as /etc/thinkfan.conf, restart, and see what happens.

This actually works. I've rebooted a couple of times and the thinkfan service always loaded at startup. I guess the only thing left is to wait for kernel version 5.8 to be able to control the second fan. Thanks for the suggestion.

[ratcashdev]

I know nobody cares whether I stick to Linux or not, but it's just too frustrating sometimes that you feel like giving up. You install a distro and everything feels perfect except that the damn fan doesn't shut off.

I actually ran linux on my laptop almost for a year and I simply took it for granted that in linux the fan is always running (and I did not really care about the number of fans running). It was only when I found thinkfan that I realized there are two of them, that there are kernel patches, etc. And I am pretty sure there are many other things to discover over time yet. The good news is it is stable and very good for my job. Anyway let's get this back to the topic: Good to hear it worked for you!

[lhofhansl]

Glad that is landed in the mainline Kernel.

@voidworker I hear you. Better to get some support in. This way was safer and more defensive and easier to pass the bar of the Kernel devs. Note that you cannot "get separate control back", you never had it. Only one fan could be controlled.

Now all existing tooling just works. I might now do a followup patch to allow dual control via the hwmon interface.

[sassmann]

I've requested Fedora to include 14232c6e788c platform/x86: thinkpad_acpi: Add support for dual fan control and it's now part of 5.6.19-300.fc32

[static-void]

I just wanted to mention that the patch 14232c6e788c platform/x86: thinkpad_acpi: Add support for dual fan control only adds support for the P52/P72/X1 extreme/x1 extreme gen2. I had to build the module patched with explicit support for the P50. I don't know if the author is watching this issue, but it would be extremely nice if this worked out of the box on the P50/P70/P51/P71 as well. It seems a bit of an odd omission given that other patches here have the code for adding support for those machines. My patch is as follows (Ubuntu 20.04, kernel 5.4.0-37):

+++ ../linux-source-5.4.0/drivers/platform/x86/thinkpad_acpi.c  2020-06-26 01:31:54.721961000 +0100
@@ -8325,11 +8325,19 @@

    switch (fan_control_access_mode) {
    case TPACPI_FAN_WR_ACPI_SFAN:
-       if (level >= 0 && level <= 7) {
-           if (!acpi_evalf(sfan_handle, NULL, NULL, "vd", level))
-               return -EIO;
-       } else
+       if (((level < 0) || (level > 7)))
            return -EINVAL;
+       if (tp_features.second_fan) {
+            if (!fan_select_fan2() ||
+                !acpi_evalf(sfan_handle, NULL, NULL, "vd", level)) {
+                fan_select_fan1();
+                pr_warn("Couldn't set 2nd fan level, disabling support\n");
+                tp_features.second_fan = 0;
+            }
+            fan_select_fan1();
+        }
+        if (!acpi_evalf(sfan_handle, NULL, NULL, "vd", level))
+            return -EIO;
        break;

    case TPACPI_FAN_WR_ACPI_FANS:
@@ -8346,6 +8354,16 @@
        else if (level & TP_EC_FAN_AUTO)
            level |= 4; /* safety min speed 4 */

+       if (tp_features.second_fan) {
+            if (!fan_select_fan2() ||
+                !acpi_ec_write(fan_status_offset, level)) {
+                fan_select_fan1();
+                pr_warn("Couldn't set 2nd fan level, disabling support\n");
+                tp_features.second_fan = 0;
+            }
+            fan_select_fan1();
+
+        }
        if (!acpi_ec_write(fan_status_offset, level))
            return -EIO;
        else
@@ -8772,6 +8790,13 @@
    TPACPI_QEC_IBM('7', '0', TPACPI_FAN_Q1),
    TPACPI_QEC_LNV('7', 'M', TPACPI_FAN_2FAN),
    TPACPI_Q_LNV('N', '1', TPACPI_FAN_2FAN),
+   TPACPI_Q_LNV3('N', '1', 'D', TPACPI_FAN_2FAN),  /* P70 */
+   TPACPI_Q_LNV3('N', '1', 'T', TPACPI_FAN_2FAN),  /* P71 */
+   TPACPI_Q_LNV3('N', '1', 'E', TPACPI_FAN_2FAN),  /* P50 */
+   TPACPI_Q_LNV3('N', '1', 'U', TPACPI_FAN_2FAN),  /* P51 */
+   TPACPI_Q_LNV3('N', '2', 'C', TPACPI_FAN_2FAN),    /* P52 / P72 */
+    TPACPI_Q_LNV3('N', '2', 'E', TPACPI_FAN_2FAN),    /* X1 Extreme (1st gen) */
+    TPACPI_Q_LNV3('N', '2', 'O', TPACPI_FAN_2FAN),    /* X1 Extreme (2nd gen) */
 };

 static int __init fan_init(struct ibm_init_struct *iibm)
@@ -8813,8 +8838,7 @@
                fan_quirk1_setup();
            if (quirks & TPACPI_FAN_2FAN) {
                tp_features.second_fan = 1;
-               dbg_printk(TPACPI_DBG_INIT | TPACPI_DBG_FAN,
-                   "secondary fan support enabled\n");
+               pr_info("secondary fan support enabled\n");
            }
        } else {
            pr_err("ThinkPad ACPI EC access misbehaving, fan status and control unavailable\n");

[lhofhansl]

@static-void , what's the difference?

My patch has this in there: + TPACPI_Q_LNV3('N', '1', 'E', TPACPI_FAN_2CTL), /* P50 */

Also @sassmann tested with a P50. So you must have picked up the wrong patch somehow.

Here's the official Kernel patch: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.8-rc1&id=14232c6e788cb1f7b96dbd08b077f90923324b24

[static-void]

@lhofhansl thanks for getting back to me, and apologies, you're right, I have picked up an earlier version of the patch https://patchwork.kernel.org/patch/11474859/

It's great that this has gone upstream. Hopefully it can get an official backport in debian/ubuntu as well. I am hoping that this solves overheating issues I am having with my P50 in the current heatwave in the UK!

[medwatt]

I just installed the latest kernel (5.8rc5) and both fans on my P50 are detected. My P50 is finally silent when it's not necessary for the fan to run like crazy. Just to be on the safe side, is an average temperature of 55C something fine, or it's better to always maintain it below 50C ?

[vmatare]

@medwatt Whether 55°C is fine totally depends on what piece of hardware you're talking about. For a CPU core that's pretty low, but for a mechanical HDD it's on the high side. Also for CPUs and other chips it depends a lot on where the temperature is actually measured. You can have internal sensors close to (or on) the DIE, which quickly jump to 90°C and above, or sensors on the PCB below the CPU, or even on the heat spreader. Those all mean totally different things.