search

perfsonar / toolkit

perfSONAR Toolkit distribution environment scripts and GUI
Apache License 2.0
31 stars 6 forks source link

Enable performance CPU governor #400

Open laeti-tia opened 4 years ago

laeti-tia commented 4 years ago

With some hardware, BIOS or OS settings, the CPU of a perfSONAR node can run under a CPU governor that will not provide the best performance to run CPU intensive tests, like high bandwidth throughput or high packet rate (O|T)WAMP tests. The CLI tools cpupower (on CentOS) or cpufrequtils (on older Debian/Ubuntu) can change that setting, but it doesn't persist after a reboot.

I think the pS toolkit, as part of its tuning scripts, should check and change the CPU governor of the OS it is installed on in a permanent manner. On CentOS this is done through a tuned configuration. See https://www.certdepot.net/rhel7-get-started-cpu-governor/ and https://www.certdepot.net/rhel7-apply-tuning-profile-server/ Under Debian we should check out this page: https://wiki.debian.org/CpuFrequencyScaling

This could also provide a benefit to some pScheduler high demanding operations and help in situations where tasks are being marked as overdue, missed or failed.

laeti-tia commented 4 years ago

tuned profiles are changing more than just the CPU governor, some tunable kernel settings related to memory usage and process scheduling are also affected. I guess these could benefit perfSONAR too.

After reading and comparing the different profiles available on CentOS in /usr/lib/tuned/ I think I'd generally recommend using the profile network-latency for perfSONAR. We could also make and ship a perfSONAR specific profile and do a mix between network-latency and network-throughput to have tuned also set the TCP buffer sizes.

If we were to consider all the variety of deployments perfSONAR runs on, from high-end servers to small nodes, but also some VM, it's then difficult to recommend a specific profile. We could have different ones per type of machine perfSONAR gets installed on (lots of RAM, limited disk speed, high-speed NIC, etc.). Maybe we should have a page in our documentation describing the different profiles available and giving advice to users to switch profiles depending on their installation type.

A lot more info at https://tuned-project.org/

laeti-tia commented 4 years ago

tuned is not available for all Debian based distros. It has been introduced with Debian 10 and Ubuntu 18.

For Debian 9 and Ubuntu 16, only the CPU governor setting can be changed.