Weird throttling behavior with GTX 965M refresh on Clevo P640RF

[cabrust]

Hi, I'm observing this weird throttling-like behaviour on my system:

• Clevo P640RF with GTX965M refresh (10de:1427, GM206)
• Ubuntu 16.10 (Kernel 4.8.0)
• Nvidia blob driver 375.39
• Needs acpi_osi="!Windows 2015" to not crash in iGPU mode.

Using watch nvidia-smi or nvidia-settings, I can see power states and frequencies. Because I don't need the Intel GPU, I run prime-select nvidia and reboot. The following happens:

• Initially, the card is in its highest power state (P0 according to nvidia-smi).
• Now, I start a game or an OpenCL application.
• The card immediately drops to P8 in nvidia-smi and to 135MHz in the settings app.
• There is no way to force it to another performance state using the nvidia command line apps.
• The "Performance" setting in nvidia-settings does nothing.

Here is where it gets interesting:

• If I close the laptop's lid and open it again, the card runs at full speed (1151 MHz) and P0 level.
• There are some compositing artifacts, i.e. the windows' edges have a thick white border on them. But, performance is good.
• But, if I then logout and login again, it's down to P8 again.

I also tried nouveau because I like the render offloading, but it doesn't support Maxwell 2 reclocking yet. I still captured some performance data, because why not. Suspend doesn't seem to affect nouveau except for one crash with Unigine Valley. Another thing I tried was pcie_port_pm=off, but that didn't change anything.

glmark2

Driver	iGPU Performance	dGPU Performance	dGPU Performance after Suspend
nouveau	1969	528	527
blob	2017	898	7352

Unigine Valley

Driver	iGPU Performance	dGPU Performance	dGPU Performance after Suspend
nouveau	350	252	(crash)
blob	351	276	2110

There doesn't seem to be anything interesting in the kernel log. I do get the Argument #4 ACPI errors. bbswitch works well except for this weird behaviour. I'm not even sure if this is a bbswitch issue, but I'd like to find out and hear some ACPI gurus' thoughts. Some things that may be to blame:

• Clevo did something weird in their BIOS. Removing the acpi_osi kernel flag doesn't change the performance/standby behavior, but it prevents crashes in iGPU mode.
• NVIDIA did something weird in the vBIOS. I checked in Maxwell Bios Tweaker, but everything looks okay.
• The Windows driver does some magic. Is there a way to trace ACPI calls/ioctls in Windows?

Thanks! Clemens

[cabrust]

Another "feature": I can get the GPU to go fast by running "nvidia-smi -r" on it while lightdm is stopped. But starting it again makes it go slow again.

[cabrust]

I also found this discussion and my nvidia-smi displays the same behavior.