search

docker / machine

Machine management for a container-centric world
https://docs.docker.com/machine/
Apache License 2.0
6.63k stars 1.97k forks source link

Importing existing machines #1229

Closed prologic closed 9 years ago

prologic commented 9 years ago

Scenario: We have several or more Docker Hosts setup on various cloud providers already running some form of Linux+Docker; e.g: Digital Ocean + Ubuntu 14.04 + Docker

Is there currently a good way to "import" these hsots into docker-machine controlled states for future management without having to destroy them and re-create fresh new ones?

nathanleclaire commented 9 years ago

Hm, there is the generic driver (in master, but not released yet), which will do such an import + provision with any machine that you have SSH access to right now.

The one caveat is that if I am not mistaken, there is not complex support for cert generation + management with that right now, so it will generate the certs and store them locally on wherever you run the docker-machine create command, and the daemon will expect to be "talked to" using those certs. This would happen any time using the generic driver, so you couldn't, say, run create -d generic on one computer, and then run create -d generic on another, only the second computer will be able to talk to the daemon.

If you have ideas / suggestions on how you'd prefer to see this work, we're highly open to it :) Trying to figure this type of scenario / sharing machines on teams etc. I think will be very important for next steps.

prologic commented 9 years ago

My particular use-case is the former where I have Docker machines already running but were not created/managed with docker-machine.

tyrken commented 9 years ago

I think if you've provisioned the docker hosts with some other method, you could also create certificates manually for the server & clients from a new or existing company CA following https://docs.docker.com/articles/https/.

Then I'd like some improvements to how machine treats partial but sensible certificate file presence in #1233 to let us still use docker machine in this situation. The generic driver will I suspect still over-validate the certificate files present, somewhere inside getMachineConfig(). I suspect all the entries in config.json aren't being actually used, only the machineDir...

prologic commented 9 years ago

What I'd ideally like to see is the generic driver (if that's the one to use) be able to ssh into an existing Docker Machine and "provision" it as such with all the right certs. I this possible? This is kind of what I mean by "import" whereby minimal manual steps are required.

tyrken commented 9 years ago

That sounds like what I think "generic" might do (if it might also re-install docker), but to be clear that's not what I'm after in #1233 - which is just the client side pointer to a correctly setup docker. I want what I do as I don't see how docker-machine can do the correct creation/distribution of SSL certs for multiple users/hosts according to arbitrary internal company security standards, as @nathanleclaire suggests.

nathanleclaire commented 9 years ago

What I'd ideally like to see is the generic driver (if that's the one to use) be able to ssh into an existing Docker Machine and "provision" it as such with all the right certs. I this possible? This is kind of what I mean by "import" whereby minimal manual steps are required.

Oh yes, this is exactly what generic does.

prologic commented 9 years ago

Oh!? I'll have to try it out on master then :) I'll see how it all works out :)

ljrittle commented 9 years ago

+1 on "create -d generic"

I had a set of n machines running docker engine but not configured properly for docker-machine or swarm (they were manually upgraded from docker 1.0 to 1.6.2 as new releases came out). Was able to transform into a swarm by running n "create -d generic" commands with proper arguments and without having to re-provision the entire machine.

Please note that "docker-machine create -d generic" will reboot your target docker engine with the new configuration.

This was great because I didn't want to follow all the manual steps to create the certs, etc OR deploy swarm without security enable. Nor did I have administrative control to easily re-provision the docker engine host(s) down to the metal.

On Wed, May 27, 2015 at 4:08 AM, James Mills notifications@github.com wrote:

Oh!? I'll have to try it out on master then :) I'll see how it all works out :)

— Reply to this email directly or view it on GitHub https://github.com/docker/machine/issues/1229#issuecomment-105832249.

ehazlett commented 9 years ago

@ljrittle thanks for the feedback! I had not thought of using generic as a general purpose re-provisioner :)

prologic commented 9 years ago

This basically doesn't work for me; See below:

prologic@daisy
Fri May 29 22:24:16 
~
$ docker-machine create -d generic --generic-ssh-key=$HOME/.ssh/id_dsa --generic-ip-address=vz1.domain.tld vz1.domain.tld
Importing SSH key...
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Enter passphrase for key '/home/prologic/.docker/machine/machines/vz1.domain.tld/id_rsa': 
Error creating machine: exit status 255
You will want to check the provider to make sure the machine and associated resources were properly removed.

prologic@daisy
Fri May 29 22:31:43 
~
$ docker-machine ls
NAME               ACTIVE   DRIVER    STATE     URL                           SWARM
vz1.domain.tld            generic   Running   tcp://vz1.domain.tld:2376   

prologic@daisy
Fri May 29 22:31:50 
~
$ docker-machine env vz1.domain.tld
open /home/prologic/.docker/machine/machines/vz1.domain.tld/ca.pem: no such file or directory
hairyhenderson commented 9 years ago

@prologic - looks like your SSH key is password-protected, so create is having trouble. Maybe try with a passphrase-less key?

prologic commented 9 years ago

Perhaps this is a bug? I'll try but most of my access keys (separate to m2m keys0 are passpharse protected.

hairyhenderson commented 9 years ago

@prologic - yeah, I agree - I just logged #1291, and I have a vague idea on how I could fix it... Possible PR tomorrow or the day after...

prologic commented 9 years ago

Cool no worries; I'll test with a passpharseless key

nathanleclaire commented 9 years ago

Definitely would like to support passphrase SSH key usage.

I've filed an issue here: https://github.com/docker/machine/issues/1473

prologic commented 9 years ago

:+1:

doublebyte1 commented 8 years ago

The docker-machine create solution, with a generic driver allowed me to import an existing docker-machine into my computer, but the fact that the certificates were regenerated, it meant that it stopped working on the machine where it was created originally.

SWARM   DOCKER    ERRORS
test   -        generic   Running   tcp://services.live.geocat.net:2376           Unknown   Unable to query docker version: Get https://myaddress.net:2376/v1.15/version: x509: certificate is valid for localhost, not services.live.geocat.net

What worked for me, was to copy the folders under ~/.docker/machine/ to my local machine. Then I had to amend the paths on config.json, as in my case they were different.