Add cgroups and device support

[xSmurf]

We need cgroups and general device support for things like U2F.

[xSmurf]

Needed for https://github.com/subgraph/subgraph-os-issues/issues/215

[timthelion]

For your reference:

Runc uses the following struct to define devices.

https://github.com/opencontainers/runc/blob/52454cf90807727923b375afd7d84ac8fd523959/libcontainer/configs/device.go

The DeviceFromPath(path, permissions string) (*configs.Device, error) function in this file is useful for filling in that struct, if you know the path to a device.

https://github.com/opencontainers/runc/blob/cae7979d1fda884a0cd66cc5fb17282569a9792a/libcontainer/devices/devices_unix.go

Here are some default devices that should probably be allowed in order to get most programs to work:

https://github.com/opencontainers/runc/blob/52454cf90807727923b375afd7d84ac8fd523959/libcontainer/configs/device_defaults.go

Here is how we actually communicate with cgroups in order to allow those devices:

https://github.com/opencontainers/runc/blob/52454cf90807727923b375afd7d84ac8fd523959/libcontainer/cgroups/fs/devices.go

Basically, runc has these device structs with proper major and minor numbers (it is only the major and minor numbers that cgroups cares about) and then we write allow and deny commands to the cgroups devices file. The '*' symbol allows us to match all minor numbers.

So to do a proper whitelist, as per the last file listed, you start by telling cgroups to deny all devices:

writeFile(path, "devices.deny", "a")

https://github.com/opencontainers/runc/blob/52454cf90807727923b375afd7d84ac8fd523959/libcontainer/cgroups/fs/devices.go#L48

You then follow up by explicitly allowing the devices you want to allow:

" writeFile(path, "devices.allow", dev.CgroupString())" https://github.com/opencontainers/runc/blob/52454cf90807727923b375afd7d84ac8fd523959/libcontainer/cgroups/fs/devices.go#L52 This is just for inspiration though. As far as I can tell, the current code is actually broken and wrong :P But it has enough working elements that it could save you time some time doing research...

Tim

On 02/26/2017 08:45 PM, xSmurf wrote:

Needed for subgraph/subgraph-os-issues#215 https://github.com/subgraph/subgraph-os-issues/issues/215

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/subgraph/oz/issues/94#issuecomment-282581429, or mute the thread https://github.com/notifications/unsubscribe-auth/ABU7-DDryBTLpGGoB1J4zUYn0_62qj0rks5rgdZegaJpZM4MMfU7.

[tildelowengrimm]

Is there any workaround (ex: for specific situations like subgraph/subgraph-os-issues#215 ) or is device support comprehensively blocked on this issue? For instance: I see https://twitter.com/attractr/status/850776132307890181, but no documentation for how that's done.

[dma]

I wrote code in Oz to do it, but it isn't finished. I should get on that. What I added was support for cgroups in sandboxes (to whitelist device major/minor numbers) and some very basic udev scripts to tell Oz when devices are plugged in. Oz would then create the device nodes in the sandboxes that "subscribe" to the device in their profile.

But it wasn't complete:

• The device nodes have to match what udev creates outside, because some (all?) applications look in sysfs for information about which device node is the right one. I didn't see an elegant way to address this, maybe there is a better way.
• The udev script handles the device node creation when the device is plugged in, but what if it's already plugged in when the sandbox is launched? Oz needs to go through already configured USB peripherals and create them when subscribing sandboxes are launched. This code wasn't written at all.
• Finally, I ran into problems with cgroups and the Golang runtime. Basically, at any given time, a golang process has an arbitrary number of worker tasks created; these are for goroutines. I implemented an ugly hack to add all of the tasks to the sandbox cgroup. That effectively worked, but it was truly less than ideal and I didn't want to ship that (the problem I encountered was basically an instance of: https://www.weave.works/blog/linux-namespaces-and-go-don-t-mix)

At that point I stopped temporarily. So, still some amount of work left to do to get this delivered for users. I intend to finish it..

For the screenshot I posted, I'd manually created the nodes in the sandbox (with a restrictive cgroup device whitelist configured by Oz).