Volumes not persisted after docker stop/start (or host reboot)

[joaocc]

Following the experiments for #26, it seems that volumes are not persisted across docker restart (or host reboot). Tested on docker 1.11.2, ubuntu 16.04.

$ docker run -d --name="docker-plugin-volume-local-persist" --restart="always" \
    -v "/run/docker/plugins/:/run/docker/plugins/" \
    cwspear/docker-local-persist-volume-plugin:latest

$ docker volume create -d local-persist \
    -o mountpoint=/tmp/test-persist
    --name=testvol

$ docker volume ls
DRIVER              VOLUME NAME
local-persist       testvol

$ service docker restart

$ docker volume ls
DRIVER              VOLUME NAME

Any ideas? Thx

[joaocc]

Did some additional testing. Since I didn't really understand why (in the readme.md example) the plugin was launched with a specific volume folder there, I decided to test if that would solve this...

$ docker run -d \
    --name="docker-plugin-volume-local-persist" --restart="always" \
    -v "/run/docker/plugins/:/run/docker/plugins/"  \
    -v "/tmp/test-persist:/tmp/test-persist"  \
    cwspear/docker-local-persist-volume-plugin:latest

$ docker volume create -d local-persist \
    -o mountpoint=/tmp/test-persist
    --name=testvol

$ docker volume ls
DRIVER              VOLUME NAME
local-persist       testvol

$ service docker restart

$ docker volume ls
DRIVER              VOLUME NAME

[tdobrovolny]

I have the same issue :-(

Probably this log line describe the issue:

open /var/lib/docker/plugin-data/local-persist.json: no such file or directory

Inside container does not exists /var/lib/docker path. When I create dir /var/lib/docker inside container issue is gone.

docker run --detach \
    --name=local-persist-volume-plugin \
    --restart unless-stopped \
    --volume=/run/docker/plugins/:/run/docker/plugins/ \
    --volume=/srv:/srv \
    cwspear/docker-local-persist-volume-plugin

Server Version: 1.11.2 Storage Driver: devicemapper Pool Blocksize: 65.54 kB Base Device Size: 10.74 GB Backing Filesystem: xfs Data file: Metadata file: Data Space Used: 6.574 GB Data Space Total: 15.03 GB Data Space Available: 8.458 GB Metadata Space Used: 11.4 MB Metadata Space Total: 16.78 MB Metadata Space Available: 5.382 MB Udev Sync Supported: true Deferred Removal Enabled: true Deferred Deletion Enabled: true Deferred Deleted Device Count: 0 Library Version: 1.02.110 (2015-10-30) Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local local-persist Network: bridge null host Kernel Version: 4.4.0-28-generic Operating System: Ubuntu 16.04 LTS OSType: linux Architecture: x86_64

 [
    {
        "Id": "sha256:4d2ff31c30e5f3b0461e9bd798b5f36c48764c11277bd544d25b790256fd6877",
        "RepoTags": [
            "cwspear/docker-local-persist-volume-plugin:latest"
        ],
        "RepoDigests": [],
        "Parent": "",
        "Comment": "",
        "Created": "2016-06-09T23:33:54.611268157Z",
        "Container": "5b6fddabd689dc608a6695f31ab9126c93dfd5e9e8c81631b0efa5e6c7490122",
        "ContainerConfig": {
            "Hostname": "6250540837a8",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "VERSION=v1.2.1",
                "ARCH=amd64"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) CMD [\"dumb-init\" \"/usr/bin/docker-volume-local-persist\"]"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:59af355db09536b2e122efaa4fb6300b2ed38b14615506c9dbf82c767b9d091a",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": [],
            "Labels": {}
        },
        "DockerVersion": "1.11.1",
        "Author": "",
        "Config": {
            "Hostname": "6250540837a8",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "VERSION=v1.2.1",
                "ARCH=amd64"
            ],
            "Cmd": [
                "dumb-init",
                "/usr/bin/docker-volume-local-persist"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:59af355db09536b2e122efaa4fb6300b2ed38b14615506c9dbf82c767b9d091a",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": [],
            "Labels": {}
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 144614003,
        "VirtualSize": 144614003,
        "GraphDriver": {
            "Name": "devicemapper",
            "Data": {
                "DeviceId": "205",
                "DeviceName": "docker-252:3-8409312-ea63f178c4e702833b6fd03a6b79a658e536779d10fcbccd936d0c73f51f655e",
                "DeviceSize": "10737418240"
            }
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:42755cf4ee95900a105b4e33452e787026ecdefffcc1992f961aa286dc3f7f95",
                "sha256:21446523cc7fbe04d340ecef022c47fdc19770147d854d9c4c1e92d67515c3ae",
                "sha256:8fc8fdb9dc0c32a98f51b969ed564e05a012fac34fbc673bf6d2e5a13d357a89",
                "sha256:648073022be34e112d109f63c58cfcf250eca2193685341a69527aa039f2a679",
                "sha256:4595aadef4755ed565041f016b486ac217092c3c68d9cdc718e4a72e377d31af"
            ]
        }
    }
]

[CWSpear]

@joaocc so the /tmp dir is for temporary files. Depending on your system and some settings, /tmp may be cleared out on every reboot.

You should probably use a different folder, cuz it will get cleared out eventually.

However, there may be another issue, so that alone may not immediately affect you, but I think @tdobrovolny is on the right track about a missing path that I will look into.

[CWSpear]

Ok, so I remember how it recreates volumes was changed slightly, and it was all good for running the plugin as a binary, but messed with running it from a container!

So thanks to @tdobrovolny for pointing out the right path in #34, there is a new Docker image out there that is half the battle. The other half of the battle is adding another -v flag when you create the plugin container. Please see the changes to the README here: 0a4d264d2ebce59171cf116b3df8aac90c93a92f

[joaocc]

Thanks! https://github.com/CWSpear/local-persist/commit/0a4d264d2ebce59171cf116b3df8aac90c93a92f is a great solution!