msm8998 multi-thread performances with updated kernel

[roberto-sartori-gl]

Hi!

I understand this may not be the correct place, and I don't even have a sony device. Feel free to close.

The situation is: I'm using a msm8998 device with 4.14 kernel based on the aosp/LA.UM.7.1.r1 branch from this repo. The device is working fine, but some users noticed an issue (that is present on all devices - it's not random). The multithread performances are noticeably worst compared to the vendor kernel (4.4). Some screen from some bench apps (left 4.4, right 4.14) (same phone, but different configured dpi).

4.4	4.14

4.4	4.14

As you can see, multi thread benchmarks can be 30% slower (or more). This is the case in all benchmarks/apps that uses multi thread.

This is becoming noticeable also in real world usage with certain apps, because the SoC has now 6/7 years, and some apps can be quite heavy (some users reported that certain photo editing apps are quite slow on 4.14, and they are much faster on 4.4).

Interestingly, the cpu single core performances are the same. The GPU performance is also the same (not present in the screenshots).

So, I'd have two questions:

• is this the case also with sony/yoshino devices?
• if so, which component may cause this? I tried changing something in the kernel device tree, but I did not improve the situation (I was able to make it worst though :) )

Of course, as the single core performance may suggest, the cpu frequency is the same with 4.4 and 4.14 (same max/min, cpu seems to scale correctly on both, cpu is at 800% (100% with 8 cores) during the multi thread benchmarks). I'm also using the same scheduler, schedutil.

Thanks!

[voidanix]

I sadly cannot answer any of the questions that you have put forward due to not owning the devices you mentioned.

One thing I would be interested in is knowing if the performance penalties you are seeing were also present on earlier versions of your 4.14 port (https://github.com/sonyxperiadev/kernel/commit/6e893e5eb4597becc2d0ae9f9999b72e11bcc698 would be a great starting point): if not, maybe a kernel upgrade introduced some regression and we are able to bisect this.

[roberto-sartori-gl]

I sadly cannot answer any of the questions that you have put forward due to not owning the devices you mentioned.

One thing I would be interested in is knowing if the performance penalties you are seeing were also present on earlier versions of your 4.14 port (6e893e5 would be a great starting point): if not, maybe a kernel upgrade introduced some regression and we are able to bisect this.

I can confirm that the difference in performances is present also with 4.14.254.

I think the issue has always been there on 4.14, but I don't think I can confirm this.

Do you own a device that has been upgrade to 4.14? If you have time, you may run the benchmark anyway with the vendor kernel and with the 4.14 from this repo, this would help reduce the kernel components which may cause the issue: if you get the same difference in performance with a different device/soc, I can probably stop looking at clocks as they are SoC specific. But if your device is working fine, so this is probably device/soc specific.

[voidanix]

Do you own a device that has been upgrade to 4.14?

Nope, only a 4.14+4.19 and a 4.19-only one.

If you have time, you may run the benchmark anyway with the vendor kernel and with the 4.14 from this repo

This seems like a good idea. Are there any specific benchmark apps I should be looking at? (hopefully some without spyware like Antutu lol).

[roberto-sartori-gl]

Do you own a device that has been upgrade to 4.14?

Nope, only a 4.14+4.19 and a 4.19-only one.

If you have time, you may run the benchmark anyway with the vendor kernel and with the 4.14 from this repo

This seems like a good idea. Are there any specific benchmark apps I should be looking at? (hopefully some without spyware like Antutu lol).

Yes let's avoid AnTuTu :D For CPU benchmarks, in the screenshots above I've used CPU throttling test and passmark benchmark. Passmark doesn't show the single thread/multithread data, I think you can just use the CPU throttling test.

[roberto-sartori-gl]

After investigating on this (a lot), it appears that the problem is/was caused by the DCVS config. Maybe OnePlus (my device) has a different config compared to Sony

I'm still checking stability and temperature, but at least I get the same (actually better) results compared to the OEM kernel.

[voidanix]

Well... I (not so) recently got my hands on a tama which happens to be a device family that transitioned from 4.9 to 4.14 and can confirm your findings about CPU performance.

4.9 (stock-based LOS20) gives a CPU score of 4782 on Passmark while 4.14 (SODP AOSP11) is at 3133.

Glancing at the LMH/DCVS configs for sdm845, they do look a little off when compared to the changes that went into 4.14 for msm8998/sdm630/sdm660, so hopefully adjusting these will fix the issue here as well.

[roberto-sartori-gl]

Well... I (not so) recently got my hands on a tama which happens to be a device family that transitioned from 4.9 to 4.14 and can confirm your findings about CPU performance.

4.9 (stock-based LOS20) gives a CPU score of 4782 on Passmark while 4.14 (SODP AOSP11) is at 3133.

Glancing at the LMH/DCVS configs for sdm845, they do look a little off when compared to the changes that went into 4.14 for msm8998/sdm630/sdm660, so hopefully adjusting these will fix the issue here as well.

I have my changes here: https://github.com/roberto-sartori-gl/4.14-kernel-oneplus-msm8998/commits/perf/4.14.314/msm8998_oneplus

I think that the QTI_THERMAL_LIMITS_DCVS_LEGACY driver is working much better on msm8998 platform compared to the new driver. Ignoring the thermals, using the LEGACY one, the CPU performances are much more stable (I can see this with the CPU throttling test, using the not-legacy driver the CPU throttles almost immediately).

It needs some fixes though to avoid some crashes: https://github.com/roberto-sartori-gl/4.14-kernel-oneplus-msm8998/commit/545637f711da387b2ee85a20ed840fd4bcfb8699

[voidanix]

I have my changes here: https://github.com/roberto-sartori-gl/4.14-kernel-oneplus-msm8998/commits/perf/4.14.314/msm8998_oneplus

Interesting, maybe some of these changes could end up in this repo?

I think that the QTI_THERMAL_LIMITS_DCVS_LEGACY driver is working much better on msm8998 platform compared to the new driver.

Hmm... I think you got it wrong: you are supposed to set the qcom,plat-mitigation-disable; and qcom,legacy-lmh-enable; DTS props with CONFIG_QTI_THERMAL_LIMITS_DCVS=y in your defconfig; currently your "legacy" driver is basically a clone of the modern one in this tree due to the changes you applied on top.

AFAIU the reason this is needed now is because sm8150 and newer configure LMh through firmware, meaning that sdm845 and older must configure it correctly through the driver (handled when the legacy prop is enabled).

[roberto-sartori-gl]

qcom,plat-mitigation-disable

I have both qcom,plat-mitigation-disable and qcom,legacy-lmh-enable (they are set by kholk in msm8998.dtsi).

The new driver ' does not work' on my device: 1) As I said, using the new DCVS driver (with those flags) causes the CPU to throttle immediately when high performances are needed. This does not happen with the legacy one. 2) With low battery (<25%), random reboots start to appear. I can easily reproduce it: with the driver enabled, as soon as I start a random benchmark with ~15% battery remaining, the device reboots. As soon I switch to the legacy driver, no reboot. The reboots happen also during normal usage, but that's of course more complex to replicate.

I did not analyze the code of the drivers (as you can see, the commits are not mine) but I did a lot of testing and I can pretty much be sure that they behave somehow differently :)

[roberto-sartori-gl]

This is suppose to fix the random reboots with the new driver: https://github.com/ederekun/x-ft_kernel_oneplus_msm8998/commit/edcadfe92f2752aded7a4b7b5aa1099a20922e4a

But the performance issue is still there, so...

[roberto-sartori-gl]

@voidanix I'm doing some more research, the legacy driver is needed in my case but the device tree stuff is probably not needed. The main difference with 4.4 is this: https://github.com/sonyxperiadev/kernel/commit/67e9edbfaf59513541dd4168f4e0b920acfec3f8

While it is true that the phone may get very hot, 4.4 does not have this limit in my device. The cpu can easily get to 90°C, while the commit above limits 4.14: after setting a very high limit (95°C) performances are very similar again.

Probably it is just a question of different thermal limits and I was looking for something much more complex for nothing :)

Just to justify me, the sensors order is different between 4.4 and 4.14, so some apps (cpu throttling for example) were reporting lower temps on 4.4 just because the CPU sensors temps were actually ignored (these apps just assume that the sensors from 0 to 7 are the cpu cores sensors). After I changed that from the app, I could see that 4.4 was allowing much higher temps.

[roberto-sartori-gl]

A possible solution comparing the legacy DCVS driver with the new DCVS driver: https://github.com/roberto-sartori-gl/4.14-kernel-oneplus-msm8998/commit/c288cb3e8c5b5555e4813df894928b9009b624ec

[roberto-sartori-gl]

I'm gonna close this.

The issue was indeed related to temperature management. Reverting this: https://github.com/sonyxperiadev/kernel/commit/67e9edbfaf59513541dd4168f4e0b920acfec3f8

already improves performances a lot, however I can't test this on Sony devices so I'm not sure how it's gonna work on different devices compared to mine.