search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.93k stars 1.96k forks source link

Provide tutorials on how to run popular CNI solutions #16460

Open lgfa29 opened 1 year ago

lgfa29 commented 1 year ago

Proposal

Nomad 0.12.0 added support for the Container Network Interface (CNI) specification and multiple vendors provide CNI plugins that, in theory, are supported by Nomad but, without more prescriptive guidance, users have struggled to deploy these solutions.

Use-cases

Run popular CNI providers, such as Cilium, Calico etc. in Nomad

Attempted Solutions

Some previous discussions in https://github.com/hashicorp/nomad/issues/12120 and https://github.com/hashicorp/nomad/issues/11748

Note Issue extracted from https://github.com/hashicorp/nomad/issues/13824. Refer to that issue for more context.

suikast42 commented 1 year ago

Indeed. Escpacially there is a bigt interesset in Cilium Nomad integration.

brotherdust commented 1 year ago

Cilium would be incredibly useful!

maxadamo commented 1 year ago

if you use Puppet, you can give a try to this module: https://github.com/GEANT/nomad_cni Also available on Puppet Forge: https://forge.puppet.com/modules/maxadamo/nomad_cni It does not use Calico, Cilium, Weave and it creates VXLAN and Bridges to interconnect the CNI among the Nomad Agents.

I have had a look into Calico, and ideally it should use consul instead of etcd.

thatsk commented 1 year ago

why weave is not showing up its would be easy and much more friendly

maxadamo commented 1 year ago

Cilium would be incredibly useful!

@brotherdust It's possible to use Cilium by Cosmonic on Nomad. However, you need to have a Consul cluster, dedicated to your Nomad cluster. For instance, I share one Consul cluster across all environments (test, acceptance, production), and I use different Nomad cluster for test, acceptance and production. This is not going to work, because the key prefix of the cilium agent cannot be customized, and it will always write to the folder /cilium.

This can be solved, if the cilium agent can use the kvstore from Nomad, or if Cilium allows to customize the key prefix.

why weave is not showing up its would be easy and much more friendly

@thatsk most of the times, they need to be adapted. For instance for Cilium they had to create a custom application (called Netreap) which talks to Cilium. Maybe Weave needs to be reworked too.