Plugin Cmd with Chroot

[huseyinbabal]

TL;DR; Can I plugin initialization Cmd with chroot support?

I am trying to use chroot with plugin commands to isolate plugin processes. Assum I have plugin A and plugin B and plugin B generates confidential file that shouldn't be accessed by plugin A. To achieve this goal, I have used following notation;

...
// path is a folder like `/tmp/kubect/` that contains `kubectl` plugin binary
dir, file := filepath.Split(path)
cmd := exec.Command("./" + file)
cmd.Dir = "/"
cmd.SysProcAttr = &syscall.SysProcAttr{
    Chroot: dir,
}
plugin.NewClient(&plugin.ClientConfig{
    Plugins: pluginMap,
    Cmd:              cmd,
    AllowedProtocols: []plugin.Protocol{plugin.ProtocolGRPC},
    HandshakeConfig: plugin.HandshakeConfig{
        ...
    },
    ...
})
...

This was my first try and I got fork/exec ./kubectl: operation not permitted.

I ran above command with sudo permissions and this time I received following generic error

Usually means
  the plugin was not compiled for this architecture,
  the plugin is missing dynamic-link libraries necessary to run,
  the plugin is not executable by this process due to file permissions, or
  the plugin failed to negotiate                                                               │

After more research, I saw following usage which manually calls chroot before using Chroot in cmd call

if err := syscall.Chroot(dir); err != nil {
     return fmt.Errorf("failed to chroot: %w", err)
}

However, this does not sound feasible since, this command is changing chroot globally and no way to pass as Cmd parameter to plugin initialization.

Any suggestion/idea about chroot usage would be helpful, and also this can be a good use-case for future users.

Thanks in advance

[tomhjp]

👋 hi @huseyinbabal - it's not an exact solution for your request, but since v1.5.0, go-plugin accepts custom command runner implementations by providing a RunnerFunc 1, which returns an implementation of runner.Runner 2. You could provide your own chroot version of the internal cmdrunner package implementation, or if it fits your use-case you could also try the plugincontainer library 3 which provides a containerised command runner. It could do with a bit more documentation, but you can see what the usage looks like in container_runner_test.go 4.

[tomhjp]

I'm going to close this for now - I think the runner.Runner interface provides a nice sustainable solution where users can provide their own implementation for specialised requests like this one, but I'm very happy to take feedback on that assertion. Also happy to review additions to the interface if they enable significant new use-cases and we can make them in a backwards-compatible manner.