search

cloudfoundry-incubator / kubecf

Cloud Foundry on Kubernetes
Apache License 2.0
115 stars 62 forks source link

C2C Networking: Diego #1256

Open fargozhu opened 4 years ago

fargozhu commented 4 years ago

Summary

The C2C communication would allow apps to communicate with each other directly.

Intended Outcome

The C2C should allowed the direct communication between app instances when networking policies are defined, in a security way (TLS enabled). It should also support app service discovery that allows apps to find each other internal addresses.

3rd party components may be plugged where necessary (CoreDNS instead of BOSH-DNS as an example).

Note

The C2C networking on Diego is a well known core feature that is missing on KubeCF.

JimmyMa commented 4 years ago

@viovanov maybe below could work for the internal app domain resolving:

  1. install bosh-dns (have it listen on port 53) and bosh-dns-adaptor into one POD, naming it as c2c-dns-adaptor, and create service (port 53) for these PODs
  2. config the coredns with below, this could be implemented with this feature: https://github.com/cloudfoundry-incubator/quarks-operator/issues/1036
    forward  apps.internal <c2c-dns-adaptor-service-IP>

Then the internal app domain name resolving flow is: app queries domain, like app1.apps.internal, this query goes to coredns, coredns forward this query to bosh-dns, and bosh-dns forwards this query to bosh-dns-adaptor.

fargozhu commented 4 years ago

@viovanov maybe I am missing something (most probably), but way would we need both BOSH-DNS and coreDNS on the same setup?

What's wrong with the flow

+---------+         +---------+         +-----------+            +------------+
|         |         |         |         |           |            |            |
|   App   +-------->+  BOSH   +-------->+  BOSH     +----------->+  Service   |
|         |         |  DNS    |         |  Adapter  |            |  Disco^ery |
+---------+         +---------+         +-----------+            +------------+
viovanov commented 4 years ago

BOSH DNS requires the BOSH director, a dependency we can't satisfy. @JimmyMa the problem with that solution is that there can be more than one internal app domain. We have found a solution for this, with what we call "App DNS" - a "CoreDNS based" replacement for the BOSH DNS Adapter.

troytop commented 3 years ago

Can we get a status update on this, and close if appropriate?

fargozhu commented 3 years ago

@troytop it's GA which means available to the public. It was not my plan to close it from the roadmap board but to set the right label.