ulatencyd does not support setups with multiple cgroup controllers merged into single hierarchy

[gajdusek]

Hi,

There was request to make ulatencyd working with cgroups setups with multiple subsystems (controllers) merged into one hierarchy (issue #45) or even with all subsystems merged into single hierarchy (issue #26).

I am going to present here my opinion which has rather universal than ulatencyd scope.

I have suspicion that every one proposing any such setup misses or ignores the fact that one subsystem cannot be attached to more hierarchies and therefore such setups restrict the cgroups usage for any one else interested.

Such setups pull restrictions on applications how and for what purpose they may use cgroups. On the other side, having mounted every hierarchy with just single subsystem controller is universal; it does not restrict anyone. I am not sure about performance but if this would have any noticeable overhead then syscalls should be implemented first. Beside this, I cannot imagine any reason for which any application in any universal distribution should require multiple controllers inside one hierarchy. IMHO, if such application does exist, then such requirement is its bug, if does not, then it is bug in the code that is mounting cgroups this way or bug in policy that requires this. The only reasonable usage I am able to imagine is for system where cgroups are used for particular single purpose and not used by other user space applications either at all or with just the very basic functionality. Even here the benefit is very small and is not matter of functionality but tiny gain in comfort.

Because, from perspective of application assuming multiple subsystems in some hierarchy, how doable is to adopt self and create and possibly track two cgroups in two hierarchies if it needs to use two controllers? I think it is very doable and even if it is not, the app may use libcgroup or similar library to make this transparent. This I call bug fixing.

And from the other side, how doable is it to adopt application that needs, for example, to account/limit cpu usage for some set of tasks that are different from a set used to partition cpu usage, if cpu and cpuacct subsystems are merged inside single hierarchy? It is not doable. This I call being pushed into a war.

So this is a question of whether ulatencyd should support only one setup, which is universal and suitable for all possible situations, or whether it should support setups, which cripples cgroups functionality but have no benefit per se.

Generally if a subsystem would have no sensible usage except being attached to another one, then the place where they should be merged is the Linux kernel, not user space. And if it is not this case then one application should not be allowed to dictate what cgroup subsystems others are allowed to use.

It is straightforward to implement these in ulatencyd and I see no technical obstacles, except the need to write scheduling rules (mappings) for every combination of merged subsystems, which would be rather hard to optimize and even if optimized the ulatencyd benefit will be lower or even none, depending on what controllers will be merged.

But I don't want implement this because want to be consistent in attitude that I will not support oppressions.

Anyway if somebody proposes that mounting multiple controllers in one hierarchy is ok, then he should be consistent too and directly propose mounting of all subsystems into single mount point as in issue #26.

I have though about this all and my conclusion is to not support setups with multiple controllers inside single hierarchy. If anyone else want work on this, I may help but will not begin of my will. There is also possibility I have missed something, but currently it seems clear to me.

Therefore only supported setup is a cgroups hierarchy for each subsystem, e.g.:

/sys/fs/cgroup
├── blkio
├── cpu
├── cpuacct
├── cpuset
├── memory
etc.

[k2s]

On Arch Linux there is default JoinControllers=cpu,cpuacct net_cls,net_prio in /etc/systemd/system.conf.

Add empty JoinControllers= to solve incompatibility with ulatencyd.

I added this instruction to https://github.com/poelzi/ulatencyd/wiki/Faq#id5

[rihardsk]

The above fix doesn't work in Fedora 20 with systemd 208. Setting JoinControllers= and DefaultControllers= has no effect - starting ulatencyd gives:

** (ulatencyd:2726): ERROR **: Multiple cgroup subsystems are mounted in single hierarchy.

It seems that it's the same issue as described here on AUR , that is, systemd has broken something.

Can somebody give any comments on this? Has this been reported to systemd devs? A quick search on their bug tracker didn't reveal anything.

[tpruzina]

@rihardsk It was reported to systemd devs, but they don't support 'writing to /sys/fs/cgroup directly', e.g. I think that systemd provides some interface to work with their cgroups, but even since mr. Lennart Poettering gave me 'middle finger' on irc, I haven't investigated further.

I think 197 version of systemd was the one that still worked, but they definitely broke ulatencyd in recent releases.