Clone of partitioned SF volume mounted by container of same OS type/version is empty

[scaleoutsean]

This is similar to #96 except that a cloned partitioned volume can be mounted, but it's empty.

1) Create a SolidFire volume and present to host (Ubuntu 16.04) 2) Partition, format and mount it (/dev/sdb1 type ext4 (rw,noatime,discard,data=ordered)) 3) Clone this volume and reassign it to "docker" account 4) On the same host, install netappdvp driver, start a container of same (Ubuntu 16.04) OS. Inside of the container, the cloned volume is /dev/sdd is type ext4 (rw,relatime,data=ordered), but it only contains lost+found.

I used "ubuntu:16.04" and the latest NetAppDVP driver. Docker host info:

# docker --version
Docker version 17.06.2-ce, build cec0b72
# uname -a
Linux ubuntu 4.4.0-93-generic #116-Ubuntu SMP Fri Aug 11 21:17:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
# docker volume inspect clone-dev-2
[
    {
        "Driver": "netapp:latest",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/plugins/a80ce5e26f2780a223296c6871f6308dc076eecd94f2bef7193a9f3203f55dc8/rootfs",
        "Name": "clone-dev-2",
        "Options": {},
        "Scope": "global",
        "Status": {
            "Snapshots": null
        }
    }
]

ext4 volume created on host, which I later clone:

Disk /dev/sdb: 186.3 GiB, 200000143360 bytes, 390625280 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xda42d1ac

Device     Boot Start       End   Sectors   Size Id Type
/dev/sdb1        2048 390625279 390623232 186.3G 83 Linux

I used fdisk defaults to partition it, and mkfs.ext4 defaults to format it.

[j-griffith]

@scaleoutsean yeah, that's how it goes for now. It's possible to add some features to help with this, but you're not going to be able to mount more than one partition and pass it into the container. Honestly my recommendation has always been "don't mix and match your schemes".

Things like clone aren't necessarily intended for "importing data into your Docker env" but rather duplicating/cloning data sets that were created using the plugin.

[scaleoutsean]

you're not going to be able to mount more than one partition and pass it into the container.

Sorry, I'm not sure I understand the explanation. Does that mean I should be able to mount the clone from within the container if i unmounted the original volume from the host? Is this use case like the old "cloned volume, identical device UUID = confusion" problem?

To be clear, I created a volume, partitioned and mounted it on the Docker host. Then I cloned with with SF PS Tools. After that I created a container and tried to mount the clone volume from within the container, which resulted in empty volume mounted in the container.

I don't really need this functionality, it's just for testing. I can start another VM to run Docker and use the cloned volume. But will I be able to mount multiple clones in different containers in this second system?

[j-griffith]

@scaleoutsean Nah... so the challenge is to use partitions or not to use partitions. nDVP could have either chosen to:

1. Always create a partition and use part 1
2. Don't create partitions and just use the base sd device

We went with option 2, so that means if you create, attach and partition a volume on your own (on any host, docker or otherwise) nDVP doesn't know anything about partitions. We just in turn mount the base device. So you're partitioned dev files are there but we don't mount them.

I know it seems like it would be easy to deal with this, but the problem is what do you do when you have multiple partitions? The way the Docker code works is you just pass in a single mount point, you don't have the ability to pass in multiple mounts for a single volume (well you could but you'd treat each partition as an independent volume and things get pretty confusing).

So, my point was that the intent of clone was to be able to clone other volumes that were created with nDVP, that way everything aligns. If you wanted to clone volumes created externally then they have to match the partitioning scheme (ie no partitioning).

It's certainly worth a feature request to make this work, but I'm not sure how high up the list it would rank.

[scaleoutsean]

Oh, I see. Thanks for the explanation. I thought it may be related to partitioning, but I expected if that was the problem the volume couldn't be mounted at all. I was surprised (and still am) that nDVP can actually mount the disk and pretend the file system is OK. I thought to troubleshoot further by mounting the same clone on the host to see if I could still read it and determine which partition was accessed by the container, but I didn't have time for that step.

I'll try without partitions to see how that works. Personally I wouldn't ask for the ability to handle partitions - IMO just a note in the docs on that would suffice. My intended scenario was "use the physical host to prepare volumes, and use containers (elsewhere, but I had just 1 VM so it was on the same host) to run workloads that use data stored on clones of that volume", by the way.

Edit 1: It seems easier to prepare data for container use outside of container environment. Maybe it'd be nice add partition number option to docker volume mount options, and initially support just 0 (raw volume; default) and 1, so that default behavior doesn't change). Edit 2: I did the following experiment on EOS v9.0: one raw and one partitioned volume (both ext4, physical host). Download a ton of same source code to both volumes - dedupe factor goes to from 1.22 (single raw volume) to 2.22x. So it appears that different partitioning approaches to Docker and non-Docker volumes doesn't negatively effect efficiency, at least for this data sample.