search

docker / machine

Machine management for a container-centric world
https://docs.docker.com/machine/
Apache License 2.0
6.63k stars 1.97k forks source link

Docker Machine is now in maintenance mode #4537

Open shin- opened 6 years ago

shin- commented 6 years ago

As has been obvious for some time now, we've slowly stopped implementing or accepting new features for the project. Its desktop usage has mostly been supplanted by our Docker Desktop product. Provisioning on a variety of Cloud providers is overall better achieved using infrakit. Overall, pursuing active development on the project doesn't make sense anymore at this point, which is why we're officially closing the faucet for non-bugfix changes, starting today.

I'm sure many will want to chime in on this, please keep the discussion civil and keep it inside this thread so we can keep things manageable.

JJayet commented 6 years ago

Oh :(

Well this was nice while it lasted :) Thx everyone for the good work !

publicarray commented 6 years ago

I would ask that you also update the readme.md

frebib commented 6 years ago

If official support for machine is closing, what is the likelihood for continued community-driven support in this repo?

SvenDowideit commented 6 years ago

for those of you needing machine, there is some activity in the https://github.com/machine-drivers organisation, and it may make sense for you to work on, and release from https://github.com/machine-drivers/machine...

gbraad commented 6 years ago

We have already looked into adding patches to this organisation, as they seem to be held from being merged here: #4509 (This is blocking for localized versions of Windows). Best is to move ahead with some form of releases, however for us: minikube and minishift we only need to link against a library.

justinclift commented 6 years ago

@shin- As a thought, the new user "Getting Started" docs still use docker-machine as a central part of the intro.

For people interested in updating the docs, which should they walk people through instead?

efrecon commented 6 years ago

I think that this really is a shame. The real power of machine was somewhere between the simplicity of getting things work on one machine using Desktop and the complexity behind infrakit. machine was perfect to fire up a few machines for testing distributed workloads at smaller scale.

gilbsgilbs commented 6 years ago

Hi!

Does somebody have an alternative software for Linux? I don't want to run docker as root on my host machine and docker-machine was giving some isolation in this respect. Is there any plan for Docker for Linux?

Thanks!

shin- commented 6 years ago

@gilbsgilbs You can still use docker-machine as you currently do!

gilbsgilbs commented 6 years ago

@shin- Thanks for your suggestion. I'm starting a new project, so using docker-machine for it would be a weird move, wouldn't it?

justinclift commented 6 years ago

@shin- Weelll... being closed to PR's kind of means using it for new projects is probably a bad idea. :wink:

shin- commented 6 years ago

@justinclift I don't want to get into too much detail because we've got more info coming in a prepared statement, but as I tried to state in the original post, the project is not closed to PRs; we're simply looking to limit those to bugfixes as opposed to new features. If the featureset of the current iteration of docker-machine fits your needs, there is no reason to abandon it, even for new projects.

Vanuan commented 5 years ago

It looks like infrakit is not active either. There were no releases for more than a year, no update on DockerCon 2018, no user documentation similar to https://docs.docker.com/machine/

Docker machine documentation suggests to try Docker Cloud which in turn is shutting down in favor of Docker EE (which is not generally available)

It all encourages to either fork the project or look elsewhere: https://landscape.cncf.io/grouping=landscape&landscape=infrastructure-automation&sort=first-commit

Not complaining, just describing my perspective.

justinclift commented 5 years ago

Interesting. Looking at the commit history for InfraKit, although it does receive new commits every few days, it mostly appears to be a one-man effort.

Activity seems to have dried up around April/May. Guessing people's time was redirected to other stuff.

Is that an incorrect way of looking at things?

gbraad commented 5 years ago

@shin- but many of the PRs (like bugfixes I provide to make internationalization on Hyper-V working) have not been taken for merge. This is not a good indication of "we're simply looking to limit those to bugfixes as opposed to new features" in light of "there is no reason to abandon it, even for new projects".

shofetim commented 5 years ago

@shin-

I don't want to get into too much detail because we've got more info coming in a prepared statement

Could you link to that official statement (whenever it is published?) I can't find it.

aliceminotto commented 5 years ago

I strongly recommend changing the docs so that they are up to date. My last hour was a senseless exploration of boot2docker, which clearly points to docker machine, which has a warning on its main page advising to use docker cloud as an up to date technology. This points to the Docker cloud doc description (not the migration page!!!!), which required me some googling to find out it was discontinued in May (but announced in March, so 7 months ago). Now came here to suggest removing the warning from the docker machine doc, I find out in this 3 months old issue docker machine is also being discontinued. This is just not how documentation should work. I'll revert back to some ad hoc solution, but I'd abandon docker if I wasn't already using it.

BretFisher commented 5 years ago

@aliceminotto It would help us all update outdated doc's if you could point to the sites or pages you're talking about.

docker-machine isn't going away, it's just not increasing the scope of features.

Docker Cloud isn't going away, it's just no longer used for server provisioning/management. It's still there for image building. Docker has other tooling for production servers like Docker for AWS, Docker for Azure, and DCI for Docker Enterprise.

tianon commented 5 years ago

I had forgotten entirely about the old http://boot2docker.io website and didn't realize the notice there was worded so poorly (apologies for any confusion that contributed to!) -- I've now updated (and slimmed down) that content to hopefully clarify better that boot2docker the ancient CLI tool is what was deprecated in favor of Docker Machine and that boot2docker the distribution is not deprecated but is rather in maintenance mode (same as Docker Machine).

To put that another way: new Docker releases, kernel updates, etc, but concerted attempts to keep new features/functionality to an absolute minimum to ensure continued maintainability for the few folks who can't yet transition to the better-suited Docker for Windows / Docker for Mac products or the production server tooling / solutions referenced above (Windows 7 users who can't Docker for Windows at all, Windows 10 Home users who thus can't Hyper-V, VirtualBox users who thus can't Hyper-V, etc etc).

afbjorklund commented 5 years ago

@tianon : you might also want to mention Linux users who don't want to transition to Mac or Windows...

Vanuan commented 5 years ago

@afbjorklund Why do you need boot2docker if you're already using Linux?

afbjorklund commented 5 years ago

@Vanuan : either because your distro was too old (e.g. RHEL6), or because you weren't allowed root...

Either way, transitioning to Docker Desktop is not an option - it's either Docker Engine or DIY LinuxKit ?

I would like to thank the makers of docker-machine and boot2docker, for making docker more acessible.

And with the effort of machine-drivers (for KVM), hopefully it will continue working for a while longer still

Vanuan commented 5 years ago

Why would I use server distro for desktop? And why I'm not administrator of my desktop? But somehow I'm allowed to access KVM?

It looks like you're looking for solution to run docker on KVM server? If that's the case, I'm currently exploring infrakit here: https://github.com/docker/infrakit/issues/913

But if you're only looking to run docker on Linux desktop I don't get why wouldn't you install latest Ubuntu along with Docker CE. If you want to run it in VM then do so. You can mount your home directory in VirtualBox and use docker over SSH. What's the problem here? There's no Docker for Linux Desktop because it doesn't make sense. At least to me.

justinclift commented 5 years ago

Why would I use server distro for desktop? And why I'm not administrator of my desktop? But somehow I'm allowed to access KVM?

Some people have to use whatever desktop OS their employer hands them. RHEL6 is an example that I've (a few months ago) been told about by a guy working in a stock trading place. :wink:

As a general data point, with Libvirt it (at least used to) have a concept of VM's that users could run in their account, that would be just for them. eg not accessible to other people logged into the same machine

Not sure if that was ever developed in any depth, as most of the Libvirt development effort went towards "system level" VM things instead of user level.

if you're only looking to run docker on Linux desktop I don't get why wouldn't you install latest Ubuntu

Quite a few people dislike Ubuntu for one reason or another. :wink:

AkihiroSuda commented 5 years ago

Starting with v18.09 (DOCKER_HOST=ssh://), setting up remote Docker machines without docker-machine is really trivial: https://medium.com/lucjuggery/docker-tips-access-the-docker-daemon-via-ssh-97cd6b44a53

Vanuan commented 5 years ago

As a general data point, with Libvirt it (at least used to) have a concept of VM's that users could run in their account, that would be just for them. eg not accessible to other people logged into the same machine

Yes, its called qemu://session Latest GNOME even has a nice app for that - Boxes: https://en.wikipedia.org/wiki/GNOME_Boxes

User mode KVM virtualization has some drawbacks wrt networking. So I think Virtualbox is the only choice in those conditions.

And to have CLI for VirtualBox the choice is vagrant. You only need some Linux distribution to run docker on top of it. And the most battle tested is Ubuntu/Debian. You can download any other distro though. But you have to package it yourself to use with Vagrant: https://www.vagrantup.com/docs/virtualbox/boxes.html

afbjorklund commented 5 years ago

Just saying that docker-machine was a good solution for those Linux users, just as it was for users of old Mac and old Windows... All that it needed was to run docker on a non-standard port, instead of hardcoded 2376 ? And a new qemu driver that didn't require libvirt group (i.e. root). Maybe infrakit/hyperkit will be an alternative in the future, but at the moment (the link above) it looks quite rough around the edges still.

@Vanuan : I know about the ubuntu/vagrant options, I just referred to it simply as "Docker Engine" above.

@justinclift : each user gets their own ssh keys/docker certs, so the machines are reasonably separated.

justinclift commented 5 years ago

@afbjorklund No worries. It's been years since I worked at Red Hat on the Libvirt team. These days I generally just use it when diagnosing issues, rather than still being super in-depth with it. :smile:

Vanuan commented 5 years ago

@afbjorklund Let's clear this up.

you might also want to mention Linux users who don't want to transition to Mac or Windows... Either way, transitioning to Docker Desktop is not an option - it's either Docker Engine or DIY LinuxKit ? I know about the ubuntu/vagrant options, I just referred to it simply as "Docker Engine" above.

boot2docker is both distribution (boot2docker.iso) and the tool to manage virtualbox (boot2docker-cli). boot2docker.iso includes Docker CE (former Docker Engine):

https://github.com/boot2docker/boot2docker/blob/d465167d83310295b5847ba315905f52c3ca1435/Dockerfile#L426-L435

And this will keep updated to new Docker CE releases.

The boot2docker-cli is gone, but in essence it's just vagrant with virtualbox. Vagrant is still around.

Docker-machine with KVM driver uses boot2docker.iso to provision Docker CE to new libvirt VMs.

Docker Desktop uses distributions built with linuxkit to provision Docker CE to Hyper-V and xhyve.

To picture it all: infrastructure

So as you can see, all the solutions include Docker Engine (currently called Docker CE for Linux) in one way on another.

There are just too many environments and virtualization/cloud solutions. Consequently there's no one tool that can work equally well on windows/mac/linux and support QEMU/Virtualbox/xhyve/Hyper-v along with different clouds and over-SSH provisioning. And such tool also needs configurability: support different ports, memory/cpu resource management, networking, etc. So probably a general purpose tool along with some configuration file downloaded over http would be the best solution.

afbjorklund commented 5 years ago

@Vanuan : yes, this what we said above. In order to replace docker-machine, you need to switch to Mac or Windows and Docker Desktop - at least until someone creates something similar with LinuxKit and libvirt...

We don't need to talk about boot2docker-cli anymore, and the support for Linux drivers has already moved to the "machine-drivers" organization - as only VirtualBox is available with the standard docker-machine.

davidhalter commented 5 years ago

@shin- If you're not allowing new features anymore, please consider adding a very clear note at the top of the README.md AND CONTRIBUTING.md.

It's quite annoying to read both of those files carefully, implementing a driver (several days of work) and not realizing that you guys will never merge any driver anymore. This should be way clearer.

If this is all a misunderstanding and I'm still allowed to pull request my driver, please let us know. It's just that at the moment it's only mentioned in a hidden file that people will see once they create the pull request.

pietrushnic commented 5 years ago

There is still no clear info about maintenance mode in README.md and/or CONTRIBUTING.md. I spent half of my holiday time to figure out working solution for ProxmoxVE VM creation and lightweight Linux for Docker deployment - I found combination of docker-machine + docker-machine-driver-proxmox-ve to work pretty good for that use case. Unfortunately it rely on boot2docker, which say it is deprecated in favor of docker-machine and maintainer on some thread suggest Rancher OS. docker-machine going maintenance mode without clearly defining what that means (saying only bug-fixes would be accepted and suggesting that it is fine for new projects is IMO in contradiction). There was also official announcement mentioned in August 2018, but no signs of any reference here.

From outsider perspective, that want to build reasonable infrastructure for SMB, docker-machine doesn't look like correct long term solution. Anyone can suggest what would be Reasonably Good™ for provisioning and managing ProxmoxVE VM as hypervisor and minimal Linux supporting Docker?

tianon commented 5 years ago

Unfortunately it rely on boot2docker, which say it is deprecated in favor of docker-machine and maintainer on some thread suggest Rancher OS.

Can you please be more specific to where boot2docker is claiming to be deprecated in favor of docker-machine so I can clarify that appropriately? (because it isn't true unless you're referring very specifically to the ancient boot2docker CLI tool that hasn't been actively maintained in years now)

The boot2docker distribution (specifically, the boot2docker.iso artifact released with every new release of Docker CE) is not going away any time soon that I'm aware of, although it does have a very narrow focus now (and thus new features/functionality are not likely to be considered for merging).

pietrushnic commented 5 years ago

@tianon You are right, I'm sorry for confusion. The message on website say boot2docker CLI - at first glance it was not clear to me if there is any difference. I'm pretty sure other users also can be confused, since most likely you will face boot2docker.iso. boot2docker CLI is something you never heard of it is hard to say what is the relation between projects. OTOH boot2docker.iso statement "maintenance mode" is vague in the same way as docker-machine, which I conclude based on this reply.

To sum up confusion:

  1. "maintenance mode" - is not clear in both projects, can I use that for production in small business?
  2. both projects lead developers/maintainers suggest use of other projects that seems to not cover all use previously supported use cases

Ideally would be to have clear statement from @tianon and @shin- if use of docker-machine is fine for production?

tianon commented 5 years ago

I can't speak for Docker Machine, but boot2docker has never been a good choice for production; its target is development / personal workstation use.

See also the added notes on https://github.com/boot2docker/boot2docker#readme, where I've tried to clarify both what we mean by maintainance mode and that b2d is not intended nor recommended for production workloads.

hamiltont commented 5 years ago

Same as many others, I was frustrated to learn about this from Github Issues, after I put effort into using machine for a SMB client

I've created a PR to update the official documentation, adding an advisory about machine's maintenance mode.

See https://github.com/docker/docker.github.io/pull/9239

pmcneil commented 5 years ago

It'd be nice to change the docker getting started too https://docs.docker.com/get-started/part4/

guidorice commented 4 years ago

edit: Found a solution which is to install docker-ce on the aws ec2 instance, and then ssh port forward the docker daemon. 

ssh -NL localhost:23750:/var/run/docker.sock -i ***.pem ubuntu@***.compute.amazonaws.com
docker -H tcp://localhost:23750 run hello-world

🎉

I started using docker-machine recently because it was installed with Docker Desktop and I realized I could have docker run on aws instances far more powerful than my local machine. The beauty is that local apps that use docker commands, like Visual Studio Code, can work with docker-machine containers just as if they were running locally.

It seems to me that docker-machine has not been superseded, so much as just there are many new ways to provision clusters including infrakit, kubernetes, etc.

I could be misunderstanding. Is there a migration strategy for doing what I described above?

roens commented 4 years ago

We use GitLab, with their GitLab Runner tool to dynamically provision EC2 Spot instances for running CI/CD jobs. GitLab Runner uses Docker Machine to perform this machine provisioning.

We've decided to move all services off (bloated) Ubuntu in favor of Amazon Linux 2. To my delight, PR #3609 would allow for this.

However, because of this "closing of the faucet", #3609 seems like it'll die in place. Please consider merging it, in its present non-conflicting, mergeable state.

alexandernst commented 4 years ago

Since the guys at Gitlab are already maintaining a fork, maybe they could be interested in taking care of this repo?

hamiltont commented 4 years ago

@usha-mandya @Dawn-Wood any update on docker/docker.github.io#9239? As a reminder, it adds advisory warnings on all Docker Machine pages. This was merged, but then reverted while the enterprise split was happening so yall could make some decisions on the future of DM. Would be good to have this advisory if DM is going to continue being in maintenance mode

AFAIK, the latest version of Docker Desktop no longer includes docker-machine

sambernet commented 4 years ago

AFAIK, the latest version of Docker Desktop no longer includes docker-machine

Just learned about this issue that way, as our internal docker-machine tooling stopped working after the update to docker for desktop 2.2.0.0

It is kind of irritating this move is not mentioned in the docker for desktop release notes either.

We heavily use docker-machine to manage and maintain shared booot2docker-based docker-machines for internal DEV and staging environments using the Hyper-V driver (so, we provision boot2docker hyper-v VMs using docker-machine). So even though we have linux and mac clients and thus use docker for windows / os x, we still heavily rely on docker-machine for our CI/CD stuff.

I'm not aware of any similar replacement for this setup - am I missing something obvious here?

BretFisher commented 4 years ago

You can always download the latest binary with brew (macOS) and directly from the repo. It's still being maintained (patches) but being slowly phased out of tools like Docker Desktop. https://github.com/docker/machine/releases

On Mon, Jan 27, 2020 at 11:47 AM sambernet notifications@github.com wrote:

AFAIK, the latest version of Docker Desktop no longer includes docker-machine

AFAIK, the latest version of Docker Desktop no longer includes docker-machine

Just learned about this issue that way, as our internal docker-machine tooling stopped working after the update to docker for desktop 2.2.0.0

It is kind of irritating this move is not mentioned in the docker for desktop release notes either.

We heavily use docker-machine to manage and maintain shared booot2docker-based docker-machines for internal DEV and staging environments using the Hyper-V driver (so, we provision boot2docker hyper-v VMs using docker-machine). So even though we have linux and mac clients and thus use docker for windows / os x, we still heavily rely on docker-machine for our CI/CD stuff.

I'm not aware of any similar replacement for this setup - am I missing something obvious here?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/docker/machine/issues/4537?email_source=notifications&email_token=AAGBNX2APIHK6CBNGSAMLDDQ74FYPA5CNFSM4FJ53G3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKAGK7Y#issuecomment-578839935, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGBNXZV7PCYP3TLWSZ7QODQ74FYPANCNFSM4FJ53G3A .

FuzzyHerbivore commented 4 years ago

I also learned that docker-machine is phased out the hard way last week 😒 On docs.docker.com it says that docker-machine is "superseded" but I can't find any information about what it is superseded by. I want to keep managing my local virtualbox machines with SOMEthing, any tips on how to do that in a future-proof way?

AkihiroSuda commented 4 years ago

export DOCKER_HOST=ssh://user@host may work for you

afbjorklund commented 4 years ago

I want to keep managing my local virtualbox machines with SOMEthing, any tips on how to do that in a future-proof way?

You can use Vagrant for that perhaps ? Or you could just continue to use docker-machine...

But you would have to come here for the binaries, since it is no longer a part of Docker (Desktop)

afbjorklund commented 4 years ago

I'm not aware of any similar replacement for this setup - am I missing something obvious here?

There is no replacement, but lots of people are interested in continued use of machine and libmachine.

Of course there are alternatives (different products), but that is not really the same thing (as the forks).

FuzzyHerbivore commented 4 years ago

@afbjorklund Well I hope docker-machine will stick around... and I would love to get it directly from here, but for some reason downloads from github are suuuuuper slow for me. The first 10-12 MB come through in a few seconds (as I expect with my pretty fast line), then it goes down to 1-2kB/s and fails eventually. I know this is kind of off-topic, but I'm confident that it's not my line, is github throttling things for some reason maybe?

justinclift commented 4 years ago

If it's not just something that's happening for you right now (only), then probably best to ping GitHub Support about it.

ilan-schemoul commented 4 years ago

I'm extremely confused about what's the preferred way to simply deploy my containers. Making containers through Docker is promoted as a standard, widely used way to create services. So that's what I used to make containers for the website I'm creating right now. But then I very obviously need to deploy these containers to my VPS. So I use docker-machine as it is the only documented, not actually deprecated, way I know of to do this. And now I learn it is in "maintenance mode" so I may not want to use it in a new project but what's infrakit ? There is 2K stars but I struggle to understand what is it for (and can it replace docker machine in a simple way) or official documentations and it is now in read-only mode (archived) so I feel like I should not even use it and there is no link in this repo Readme/issues to a new repository. Why is there no documentation and is discontinued if it replaces docker machine. Docker seems such a popular solution yet I can't find a single one non discontinued (or in "maintenance mode") way to deploy my containers. If docker is so widely used how do the possibly millions of devs working with it deal with deploying their application ??