Containerize the volume plugin

[gjonespf]

Sure you're aware of, but thought I'd voice my interest after using this plugin on a few machines and really liking it.

Documented here: https://docs.docker.com/engine/extend/plugin_api/

Has plugins running as "special" docker containers, brought up before docker proper and shut down after docker stops. Not sure of the impact of how they work on the fact you're using SMB tools/permissions/privileges. I think you're already using some of the ideas, so might not be an issue. Would make installing the plugin very simple and much cleaner however, when plugins come out of experimental.

[ahmetb]

@gjonespf we tried to containerize the driver, however the main problem is, the driver executes mount commands to mount SMB shares on Linux. When we containerize, the mounts are restricted to the container's namespace and therefore does not appear on the Linux host, thus not available to containers using the volumes. Last time I checked, this was a limitation of docker (or the Linux kernel).

If the situation has changed, we can take a look at containerizing the driver. BTW in the docs you sent, I don't see "running as special docker containers" reference. Could you please quote?

[gjonespf]

Indeed, wondered if this may be an issue. Would suggest that issue (containerized mounts) should be upstream on docker, as I'd expect having mount tools working correctly for plugins would be pretty critical for anything doing volume plugins based on mount. I raised the suggestion more as an easier way for people to install/update this driver. That being said, I'm at a loss to find much on docker plugin infra at all, and current suggestion (on the page I linked) is to run them outside of containers (as you're already doing). Re: "special" containers - I believe I read it on some Weave documentation, I'm unable to find much on plugins as containers tbh. Also unsure as to how these fit into the existing Kubernetes plugin infrastructure.

[gjonespf]

Oh, for ref here is how the Weave team is doing it. Looks like they've got separate Docker/CNI plugins. https://github.com/weaveworks/weave/tree/master/plugin Tho obviously CNI not applicable to what you're doing here. Everything I read pretty much implies to me that volume plugins are still very experimental.

[ahmetb]

Would suggest that issue (containerized mounts) should be upstream on docker

It's here:

• https://github.com/docker/docker/issues/10088
• https://github.com/docker/docker/issues/14630
• https://github.com/docker/docker/issues/17034

Looks like it is merged now. Perhaps we should give it a try. This was the only reason we could not containerize the plugin back in the day.

Everything I read pretty much implies to me that volume plugins are still very experimental.

Volume plugins support have been out for pretty long and the API has gone many revisions, I think things have settled down at this point and we have a stable mechanism.

I'm changing the title to reflect the latest here.

[gjonespf]

Nice work, yeah those items look a good reflection. Experimental - agreed, it's been around for a year or two, just wasn't seeing much documentation to help you out. Expect if the mounts work, then it's "just" a case of sorting out how to build plugin api wrapper to handle mounting etc commands.

[markvr]

@ahmetb hi, is this likely to happen please? I note the the "docker-for-azure" project has implemented this with their "cloudstor" plugin (https://docs.docker.com/docker-for-azure/persistent-data-volumes/) but that is only (officially) available if using that project, and it is closed source with minimal docs.

Enabling mounting Azure storage using docker plugin would be very useful. Otherwise it severely limits the use of Docker on Azure.

[powareverb]

@markvr as @ahmetb is now working at Kubernetes would expect that it's not high on his priority list, and don't blame him. I'm still keen on this, but have started looking at other avenues to solve same problem. Docker plugin infra is finally maturing so would suggest it's getting easier for anyone to jump in and try this stuff out.

[markvr]

I would have hoped that Microsoft could have more than one person supporting their products. I like the open source approach they've taken with a lot of things, but they seem to just abandon a lot of them as well, which makes it impossible to know if we can rely on them for production systems.

[msftgits]

This driver is no longer supported and will not be maintained moving forward. We recommend users use CloudStor for Docker native solutions.

Thanks for your contribution, but to reduce confusion, we are closing issues, pull requests, and marking the repo as 'Archive' for the time being.