Implement IPSEC mode

[eyakubovich]

For users that require network level encryption, flannel should be able to use IPSEC as the encapsulation protocol.

[jpstrikesback]

Am I correct in thinking this would obviate the need for e.g. AWS VPC? What is needed for this to happen?

[eyakubovich]

@jpstrikesback a VPC is about isolation -- making different tenants not run into each other for example in IP space. It also provides security in the sense that one tenant cannot snoop on another's traffic. But it doesn't offer encryption so if a user doesn't trust AWS not to intercept traffic at switches and inspect it, VPC doesn't help.

In practice, if you can't trust your cloud provider, network level encryption is not enough. However IPSEC in flannel can be useful in cases where a cluster spans across untrusted networks.

[jpstrikesback]

Fair enough (on network level encryption). I'm interested in this for use on providers like DigitalOcean where there is no analogy to AWS VPC (and perhaps for hybrid solutions) so I'm looking for isolation at the host level via something like Flannel. Stampede.io has implemented an IPSec VPN in a container IIRC as part of its solution (which is interesting and perhaps a stopgap...tho I guess there's no reason it can't be a robust solution if config is/was in etcd et al).

[offlinehacker]

+1

[sporkmonger]

Honestly, it's really not so much a question of not trusting your hosting provider. Hosting providers don't necessarily get a choice in the matter of whether the fiber has been spliced and tapped. It's just a good idea to encrypt any traffic you're not in absolute control over. And even if you are... it's probably still a good idea.

[runningman84]

A simple encryption like openvpn would be good enough. I would really like to span a coreos cluster between different cloud providers like linode, online.fr and others.

[mark-kubacki]

Setting up IPSEC in transport mode with the help of Consul or etcd is what I currently use. You might find it a viable (and proven) substitute.

[cornelius-keller]

Hi wmark,

is there any documentation on your consul / etcd solution available? I would like to try this, but beeing very new to ipsec I would like to have an example to follow.

[ibotty]

hi @wmark, I am pretty sure that at least (flannel using) vxlan does not work with ipsec transport mode. It should work using tunnel mode though.

[balboah]

this would be very nice as I would like to hookup different data centers across the public internets

[eyakubovich]

@MohdAhmad is currently working on IPSec backend so stay tuned.

[ibotty]

I have written a blog entry on how to do that.

But @balboah, you surely have a dedicated router on each site of your subnets, right? That should be a stock site-to-site setup. Just (transparently) encrypt traffic coming over udp port 8472.

[cornelius-keller]

Hi there,

I created a demo with an encrypted flannel network using ting. You can find it here https://github.com/cornelius-keller/coreos-vagrant-tinc As I am really new to coreos systemd etcd etc. I would like to hear your comments / suggestions improvements.

It uses tinc for the encryption, flannel for subnet allocation and etcd for distributiong the configuration. It is a long weekend hack and I am new to the whole technology stack. An attempt to dive into coreos and do something useful. But it works.

I am sure that the whole systemd services can done much more elegant by an experienced systemd user so I would love to see your comments on that or even better some pull requests.

[balboah]

@ibotty cool. Yes setting up routers on each side is an option, but at least initially I feel that peer to peer encryption where flannel sends packets to publicly routed ip's would be more flexible and not requiring high availability setups of additional routers

[diranged]

Just chiming in here -- I agree that this would be a great and obvious feature to implement. It both allows for cross-region networking models, as well as adds a layer of warm-fuzzy encryption. Any movement on this?

[eyakubovich]

@diranged Yes, we have a PR ready: https://github.com/coreos/flannel/pull/290

Please help by reviewing it and providing feedback.

[diranged]

@eyakubovich thanks for the link -- I'm not qualified to comment on the code itself, but I'm reading the PR now and will comment where I can.

[cvle]

+1

[mkutsevol]

PR #637 is the latest one regarding this issue.

[RyPeck]

See #929 for the latest work here. Still experimental.

[Iodun]

Are there any thoughts on using MACsec on top of VXLAN instead of IPsec? Available in mainline Linux kernel 4.6 and newer. https://developers.redhat.com/blog/2016/10/14/macsec-a-different-solution-to-encrypt-network-traffic/

[eliaslevy]

If considering using MACsec, then you may also want to look at WireGuard.

[jameskeane]

It's not Flannel, but Scrambler provides an ipsec mesh network on top of strongswan.

It reuses a lot of built-in stuff to stay simple: RBAC, k8s api, and cluster PKI (kubelet client certs). The mesh is created by an agent run on each node that reads the cluster nodes, and generates an ipsec config using each node's name and pod_cidr on an interval. It dynamically reconfigures on node join/leave. And it's less than 200 lines of bash!

It doesn't do everything flannel does, but it might be enough for some.

[G3ph4z]

so is this not yet implemented?

[artheus]

https://github.com/coreos/flannel/tree/master/backend/ipsec Should this issue not be closed, now that ipsec backend exists in master branch? Edit: I see now that it has not yet been released.

[zbindenren]

Will the ipsec backend be included in the 0.10.0 release?

[nautik1]

Hello guys, this issue can be closed no? I see IPSec backend was added & released in 0.11.0

Edit: I see it's still listed in the experimental backends section though

[txomon]

Yes this can be closed. Be advised that somethings like forwarding might not work though

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.