search

projectatomic / container-storage-setup

Service to set up storage for Docker and other container systems
Apache License 2.0
153 stars 77 forks source link

Add support for Docker daemon "-g" start up option to point to LVM-direct mount #249

Open bklau opened 7 years ago

bklau commented 7 years ago

Currently, docker-storage-setup assumes that the default location of the "root" filesystem for Docker images/containers to be at "/var/lib/docker". What if I want it to be at another place via "-g" option of Docker daemon to point to say, "/mnt/something" or "/var/something"?

rhvgoyal commented 7 years ago

docker-storage-setup is the one which is setting up the storage for docker. So it should be the one communicating location of images/containers to docker and not vice-a-versa.

So while we do hardcode the location, what's the use case for changing it. I am aware of "-g" option of docker, but who uses it and why and in what context.

bklau commented 7 years ago

@rhvgoyal : some folks mounted alots of host volumes to container, esp legacy softwares. So usually they want to specify the "-g" option for 1 logical volume(non-thin) and then another volume(thin) dedicated to devicemapper.

rhvgoyal commented 7 years ago

Sorry I don't understand. Can you explain it in little bit more detail.

bklau commented 7 years ago

@rhvgoyal : Assume that I have 3 devices : "/dev/sda", "/dev/sdb","/dev/sdc"

  1. The primary OS is located on "/dev/sda" , maybe 8GB, say.

  2. I want Docker's "/var/lib/docker" to be on "/dev/sdb" which has say, 100GB So i mount "/dev/sdb" to say, "/mnt/docker" and set docker daemon to start up with "-g /mnt/docker" option. I do that because "/dev/sda" do not have much diskspace(8GB) to mount a host-to-container volume: "docker run -v "/somedir":"/my_dir" nginx" where "/somedir" is located on "/dev/sda" So I used "-g" option to set the Docker root dir to "/dev/sdb" which has 100GB and leave "/dev/sda" alone for the OS.

  3. I configured "/dev/sdc" as dedicated DeviceMapper storage driver's "thin pool" location to store container and images.

Hope that's clearer.

rhvgoyal commented 7 years ago

So you are taking care of creating a filesystem on /dev/sdb and mounting it on /mnt/docker? An easy workaround is to mount /dev/sdb at /var/lib/docker/ instead? How about that?

container-storage-setup does not know what's the root location docker wants to use. May be we can introduce an option say DOCKER_ROOT_DIR which will have /var/lib/docker as default but if user has provided a separate path, use that instead.

I did not feel strongly introducing that option as it is little corner case and it is easily fixable by mounting disk on /var/lib/docker instead.

bklau commented 7 years ago

I may not want to mount at "/var/lib/docker" for some reasons.

rhatdan commented 7 years ago

you may be pushing the tooling too far. Most people will be satisfied with mounting on /var/lib/docker trying to figure out just adds a ton of complexity for an incredibly small subset of the users, who most like could figure out how to configure the system themselves.

rhvgoyal commented 7 years ago

How about using new options which allow setting up rootfs for container run time.

Add your disk /dev/vdb and /dev/vdc to a single volume say docker-vg. And specify following. DEVS=docker-vg CONTAINER_ROOT_LV_MOUNT_PATH="/mnt/docker" CONTAINER_ROOT_LV_NAME="docker-root-lv" CONTAINER_ROOT_LV_SIZE="20G"

I think this will do what you are looking for. Create a separate volume for docker rootfs and mount it at the path you are looking for.

bklau commented 7 years ago

@rhvgoyal : So CONTAINER_ROOT_LV_NAME="docker-root-lv" will used to carve out a LV from the docker_vg for the container rootfs besides the 2 DM "thin " LVs pool data and metadata??. Pls clarify. thx.

rhvgoyal commented 7 years ago

@bklau, yes. container-root-lv will be carved out and mounted at /mnt/container. And then code will go on to setup another logical volume (thin pool).

IOW, container-storage-setup has been enhanced to handle this use case where a user does not want docker's metadata storage from root file system but from a separate disk.

bklau commented 7 years ago

@rhvgoyal Thanks. Then that will work! :)

bklau commented 7 years ago

@rhvgoyal : How can i install this newer package?. The old one be installed via: "sudo yum install -y docker-storage-setup.noarch"

rhvgoyal commented 7 years ago

Package name now is container-storage-setup. what distribution and repository you are using? On fedora 25, I imagine this might be available.

bklau commented 7 years ago

@rhvgoyal : I'm using AWS Linux AMI.

rhvgoyal commented 7 years ago

@bklau I think you will have to talk to amazon for packaging new container-storage-setup in image for that. I can't do anything about that.

LongLiveCHIEF commented 7 years ago

@bklau i've packaged and hosted both rpm and deb formats for this service at https://packagecloud.io/LongLiveCHIEF/container-storage-setup. There are several different methods of installation, but the easiest (for RedHat family OS,s), is to run:

curl -s https://packagecloud.io/install/repositories/LongLiveCHIEF/container-storage-setup/script.rpm.sh | sudo bash

@rhatdan I mirror the git repo and run automated builds to this package hosting service whenever you run new releases. All the image metadata and stuff has your copyrights and licenses in there, I just have my own contact information in the maintainer metadata fields.

cueedee commented 1 year ago

For future visitors who like me are wondering what that -g option (or its --graph long form cousin) does; apparently, it is the former nomer for what is now --data-root;