Spike: Investigate an extraction layer that can satisfy capablity dependencies

[mikevanhemert]

As a SWF Engineer I need to know that Virtual Service can mask internal and external resources So that capability dependencies are really easily satisfiable

Acceptance Criteria

• Solution works with any k8s cluster / distro
• Solution can work with internal and externally deployed things (Zolando & RDS, for example)
• Solution can work with on-prem and cloud infra

[corang]

My gut feeling is to use a k8s service with an externalName pointing to the domain name of the dependency or in the case of the dependency not having a domain name then a headless service with the IP of the dependency in the definition of said headless service

[corang]

Actually headless all the time for consistency is probably better

[corang]

So here's what I came up with.

When a dependency has a domain name that is discoverable from the cluster:

apiVersion: v1
kind: Service
metadata:
  name: some-dep
spec:
  type: ExternalName
  externalName: google.com

When a dependency doesn't have a domain name or it isn't discoverable from the cluster (must have a static ip)

apiVersion: v1
kind: Service
metadata:
  name: some-dep
spec:
  clusterIP: None
---
apiVersion: v1
kind: Endpoints
metadata:
  name: some-dep
subsets:
  - addresses:
      - ip: 8.8.8.8

The ExternalName based service can point to another service in the cluster even if it's in a different namespace by using the FQDN of the service name. cluster.local must match the domain name of the cluster.

apiVersion: v1
kind: Service
metadata:
  name: some-dep-internal
spec:
  type: ExternalName
  externalName: dep-fulfillment.otherns.svc.cluster.local

[corang]

Keep in mind some protocols will not work with this approach if they depend on the domain name being correct, a common one is http/https since the host header will be the service created instead of the actual dependency's name. TLS would also be broken if the cert being served doesn't match the service's name.

[RothAndrew]

I don't see any red flags, if this works for us it is nice that we don't have to do stuff with Istio. What's the next step? Make sure this works with gitlab by testing having gitlab point at both a local postgres pod and an RDS DB with no other changes? (meaning, nothing in gitlab configuration should change when swapping between the two, it should point at the same service and the same secret for the creds, but the contents of those resources would change based on what is behind the service.

[mikevanhemert]

I like that suggestion @RothAndrew

[corang]

@RothAndrew I can definitely do the in cluster and even external to cluster tests of it but an RDS specific test I'll need someone else to do or will need assistance with