new feature: [Integration] Docker Plugin For OpenDAL

[yuchanns]

Feature Description

Docker Engine volume plugins enable Engine deployments to be integrated with external storage systems such as Amazon EBS, and enable data volumes to persist beyond the lifetime of a single Docker host

FYI: https://docs.docker.com/engine/extend/plugins_volume/

Problem and Solution

Implement A Docker Volume Plugin for OpenDAL, so users can integrate Docker with OpenDAL in this way:

services:
  app:
    image: whatever/image
    volumes: [configdata:/config]
volumes:
  configdata:
    driver: opendal
    driver_opts:
      root: "/"

Additional Context

As a volume plugin, we need to implement Volume Plugin Protocol. It is an HTTP server that registers itself so the Docker daemon can find it and use it.

While implementing plugins is language-agnostic, it is notable that Go has a plugin helper that benefits for developing.

Rclone has a docker volume plugin too. FYI: https://rclone.org/docker

Are you willing to contribute to the development of this feature?

• [X] Yes, I am willing to contribute to the development of this feature.

[Xuanwo]

Interesting. I'm not very familiar with how Docker volumes work. It seems we can implement such a volume plugin, but I'm unsure about the IO processes involved. I assume we need to mount a filesystem to a path, and then the application accesses that path directly?

[yuchanns]

I assume we need to mount a filesystem to a path, and then the application accesses that path directly?

Yes, we need to mount and access it. It relies on either fuse3_opendal or virtiofs_opendal.

[zjregee]

This looks cool, does this plugin do something similar to parsing the configuration and helping start an opendal-based file system? virtiofs probably wouldn't be an option since it can't be mounted on the host side, but fuse would work fine!

[yuchanns]

Does this plugin do something similar to parsing the configuration and helping start an opendal-based file system?

Yes. And it shares volumes across multiple containers.