Support Swarm mode / clarify its status

[chrisbecke]

Tell us about your request

Please clarify the status of Docker Swarm mode.

Which service(s) is this request for?

Docker Engine

Additional context

Docker swarm mode is an awesome container orchestrator. Swarmkit is still upstream in moby and being actively developed. However the larger zeitgeist is that swarm is deprecated and K8s should be used instead and consequently swarm mode is increasingly not being actively targeted by projects.

However, it is super simple to setup and get working compared to K8s. New features are being added to swarm(kit) like jobs - which look perfect as CI/CD runners or Server-less task runners. But, with the hesitancy that projects are showing in continuing to invest in swarm related features, I fear the world will lose the best little multi node orchestrator that could.

[ximon18]

Is this perhaps relevant: https://github.com/docker/roadmap/issues/86 ?

[chrisbecke]

nope. classic swarm was a - not sure what it was exactly actually - a kind of plug on orchestrator that presented as a single docker node. it was deprecated precisely because it was replaced with swarm mode.

[ximon18]

But doesn't that issue effectively answer this one? I.e. classic swarm is deprecated but swarm is still actively supported?

[chrisbecke]

Mirantis only committed to swarm until 2022 - understandable as they are a very K8s corporate facing solution provider and they target the space where that scale is needed.

Since that deal, every tech article covering swarm has basically stated that it's dead and its time for everyone to move to K8s. Which is stupid because swarmkit is part of moby and very much alive. But also accurate, because Mirantis got the corporate customers, who might have been using Swarm. Which means that literally no company is providing corporate level support for Docker Swarm.

However, on tonights "Docker Talks: Donnie Berkholz - Head of Product @ Docker" session, it was apparent that I am not the only person thinking that docker swarm is simple and effective and consequently deserves some attention to get it back in front of people as a viable option: there was near unanimous support amongst the chats participants for docker swarm.

So, swarm mode exists, and is actively developed. But as a devops / developer I cannot go to my boss and say it's a supported solution that we can build solutions on. And projects like OpenFAAS, which literally started on swarm, have migrated to K8s and now don't recommend swarm for much at all.

[PavelSosin-320]

@chrisbecke Everything is fine except one blocker issue: Only Windows Ubuntu-20.04 WSL distro with K8s knows to use systemd and run Podman that is the real Docker engine replacement for the loosely connected laptops used as a development machine. Without systemd Podman is hardly useful and Docker startup suffers from instability too. But this is in Microsoft's hands. RedHat already started pulling Docker support off from its Linux distros but Windows WSL CentOS8 and RHEL8 don't run Podman well. I didn't try yet other WSL distros. Ubuntu is ahead for today.

[aCandidMind]

Totally hooked on the great news that Mirantis is going to invest in swarm beyond 2022. They are expanding the SwarmKit team and adding new features (one has already hit moby master) https://www.mirantis.com/blog/mirantis-will-continue-to-support-and-develop-docker-swarm/

The future of SwarmKit is fine, and @BretFisher talks about it on his show here: https://youtu.be/L5N43aQQArw

[decentral1se]

I do wonder where we're at now? I just read https://github.com/docker/roadmap/issues/194#issuecomment-782912802 :thinking:

Still see some movement on https://github.com/moby/moby/pulls?q=is%3Aopen+is%3Apr+label%3Aarea%2Fswarm though :pray:

And then what of https://github.com/docker/roadmap/issues/194#issuecomment-855380685?

[chrisbecke]

Just a personal comment from Dockercon'21 - the talks involving swam seemed to be well populated with people using, or thinking of using, swarm in production. The consensus opinion seemed to be that swarm was, for a number of use cases, more fit for purpose than K8s :- i.e. the simplicity of swarm over K8s allows a smaller team to deliver a more secure product with higher SLOs. That said - officially there was no word on the state of the CSI volume driver support that Mirantis mentioned some time ago. Or any official word on swarm at all. So that is concerning.

[decentral1se]

A bug fix PR from Mirantis staff https://github.com/docker/swarmkit/pull/3003 :scream: What a cliff hanger!

[dperny]

@chrisbecke I'm actually working on CSI support right. Like, literally today still working on it. There have been delays, but at this point I'm hammering out the final API and CLI. It's very nearly completed.

[decentral1se]

@dperny you are a hero!!!

[tomposmiko]

Is there any update on the status of swarm? As far as I can see, it looks exactly what we need, simple, lightweight, easy to manage, easy to run, easy the whole concept.

However, we do not jump on a track that is heading dead end.

[mhemrg]

I highly recommend you to take a look at Hashicorp Nomad. It's as simple as Swarm and it's being actively maintained. https://www.nomadproject.io/

[djmaze]

Already using swarm mode for a couple of years and really love it. In my opinion it has the right layer of abstraction (i.e. almost none) in order to be flexible as well as to keep overall complexity low. (Also, I built some useful tooling that I want to release with documentation somewhere in the near future.)

I think swarm suits the needs of most small to medium enterprises as well as private users perfectly (minus some things like improved volume driver support).

That said, the problem (from my outside point of view) seems to have always been monetization of swarm as a product, as selling an enterprise version seems not to have worked out.

Most small companies probably do not have the knowledge and time to contribute to the development of swarm directly. As another kind of solution, I imagine an open crowdfunding effort where users and enterprises could spend money (recurrently at best) for further development. Would be all in for something like that. I admit of course that real "corporate level support" would be outside of scope for this approach.

(About Nomad, I don't know, did not have a closer look yet. Can it really come close to swarm's ease of use, as well as the added bonus of the configuration being compatible with the docker compose format? But probably not the right place to discuss this.)

[DavidSche]

The biggest problem for Swarm is that Docker is not willing to lose 100% of its control, Docker is not willing to invest in continuing development, continuing its life cycle, but also does not allow those other developers who love it to join the team, continue to improve it, let Swarm slowly and slowly die

[mhemrg]

I guess there are some road blockers in Swarm's initial architecture that makes continuing its development so complicated. For instance, Swarm's networking stack is so easy to use but it's a huge deal for a small team to keep maintaining it. It might be a good idea to opt-out Swarm's networking feature and let people use CNI drivers.

[chrisbecke]

all of the 3rd party CNI drivers have dried up.

[membersound]

It would be pity losing swarm mode, as it provides the possibility to simply create a zero-downtime deployment pipeline only for a single server, without having a loadbalancer or multiple machines in front. And that only by using docker swarm init, and some variant of a docker-compose.yml as follows:

    deploy:
      replicas: 1
      update_config:
        order: start-first
        failure_action: rollback
      rollback_config:
        parallelism: 0
        order: stop-first
      test: ...

[Leopere]

Swarm is wonderful I really hope that it gets carried forward. It would be absolutely catastrophic to have to find some alternative. Kubernetes is a nightmare.

[pozylon]

Swarm is wonderful I really hope that it gets carried forward. It would be absolutely catastrophic to have to find some alternative. Kubernetes is a nightmare.

There is so much potential in docker swarm mode! Imagine some love in docs, implementation of long needed features (persistent volumes, manager ip changes, affinity) and a marketing push. Kubernetes people know now what they are fighting with as memes go viral (for ex. https://twitter.com/lexzap/status/1594110738393870337?s=20&t=RUN3L2GyOBeUo4Zky6iaBg) but i'm not sure they know a simple solution already exists for years.

[Leopere]

implementation of long needed features (persistent volumes, manager ip changes, affinity)

Since Storage Area Network is a very complex problem to solve because of data availability and redundancy I don't know for sure if it seems within scope to handle swarm wide persistent volumes. May I suggest you look at GlusterFS or SeaweedFS both are very good solutions. I use Gluster currently in many long standing SAN's and am currently scouting and testing SeaweedFS for an insane side project I'm working on to handle SQL Database failover in a Swarm context.

I would love the ability to bind a port to a specific IP though, when I deploy a swarm service to a node statically and bind it using host networking I think its only natural to want to also potentially select a specific interface.

I love the idea of Kubernetes/K8's/Kwhatevers but the reality is that its so hyper diverse in its potential scope that it almost fights its own usefulness. Then when you do actually adopt it for production you end up finding scenarios where you're worrying about google cloud's own implementation of it and its restrictions and limitations and so on. Its great for scalability but it seems like this could be easily remedied with an orchestration tool like Salt Stack or literally anything and a simple Go binary equipped with a small amount of AI and Analytics to determine when to summon more resources from varying cloud providers.

[pozylon]

SeaweedFS

Agreed, the persistent volume thing is quite complex but oh wow never heard of SeaweedFS, tried all the rest (nfs, gluster, rexrays3, ...), i'll give it a try. I'm currently just replicating a shared folder across nodes with csync for failover.

[Leopere]

SeaweedFS

Agreed, the persistent volume thing is quite complex but oh wow never heard of SeaweedFS, tried all the rest (nfs, gluster, rexrays3, ...), i'll give it a try. I'm currently just replicating a shared folder across nodes with csync for failover.

I have no idea if you can reach out to someone directly on here without spilling the beans on an email address you own so feel free to reach out to me if you wanna bounce questions off someone. stuka_stormreaver.noa2m@simplelogin.com I'll disable that email in a little while but if you want to reach out to someone to discuss the woes this is potentially better than posting our private contact info on github forever.

[jonisapp]

Looks like there are some good news! Especially upcoming Swarm Container Storage Interface (CSI). https://finance.yahoo.com/news/docker-swarm-still-thriving-three-170000776.html

[Leopere]

Looks like there are some good news! Especially upcoming Swarm Container Storage Interface (CSI). https://finance.yahoo.com/news/docker-swarm-still-thriving-three-170000776.html

Very happy to hear this because if they're planning more things that's optimism enough for me to continue to support it.

[shinebayar-g]

Swarm classic definitely left a negative opinion towards Swarm. But man Swarm mode is such a joy to use. But extendibility is questionable. It looks like it doesn't embrace CNI, CSI driver support like k8s, big factor towards supporting major features, enabling third party integrations.

Looks like there are some good news! Especially upcoming Swarm Container Storage Interface (CSI). https://finance.yahoo.com/news/docker-swarm-still-thriving-three-170000776.html

I can't find anything about this on the docs. https://github.com/moby/moby/issues/39624 there is this but no actual implementation for like EBS for example? Quick doc search gives this https://docs.docker.com/engine/extend/EBS_volume/ from 2017...

Nevertheless, I'll give it a shot.

[jonaskuske]

@shinebayar-g Not in the docs because it isn't released yet. Will be part of the next (feature) release, 23.00 or something like that I think

[Leopere]

Swarm classic definitely left a negative opinion towards Swarm. But man Swarm mode is such a joy to use. But extendibility is questionable. It looks like it doesn't embrace CNI, CSI driver support like k8s, big factor towards supporting major features, enabling third party integrations.

Looks like there are some good news! Especially upcoming Swarm Container Storage Interface (CSI). https://finance.yahoo.com/news/docker-swarm-still-thriving-three-170000776.html

I can't find anything about this on the docs. moby/moby#39624 there is this but no actual implementation for like EBS for example? Quick doc search gives this https://docs.docker.com/engine/extend/EBS_volume/ from 2017...

Nevertheless, I'll give it a shot.

I wholeheartedly agree about swarm being an absolute joy to use. It really seems like a shame that it feels a bit like even with Mirantis suggesting they’re continuing to support it that it feels very much like they feel defeated by Kubernetes out of the gate even though kube very much feels many layers removed from what docker intends to be.

As far as I can tell kube is about as useful as it would feel if VMWare randomly decided to open up their tech stack without any documentation or support.

[fetwar]

Just wanting to leave this more up-to-date status here for anyone else stumbling upon this.

On 2024-01-30:

Swarm is actively maintained by Mirantis, with @dperny as lead maintainer.

Source: @corhere (Mirantis employee) comment on moby/moby#47241