search

skupperproject / skupper

Skupper is an implementation of a Virtual Application Network, enabling rich hybrid cloud communication.
http://skupper.io
Apache License 2.0
589 stars 73 forks source link

Alignment with KEP-1645? #479

Open rosenhouse opened 3 years ago

rosenhouse commented 3 years ago

Hi there,

I'm looking at various multicluster service solutions, and trying to get my bearings in this space. Skupper looks really nice.

I'm wondering if there's been any discussion within this project of aligning with the Kubernetes Enhancement Proposal for Multi-Cluster Services, KEP-1645.

If I'm understanding Skupper right, when I run skupper connect, I end up with something similar to the capabilities in that KEP -- although that KEP is written as if all namespaces on the clusters are connected in that way, while Skupper makes it opt-in per-namespace.

If that understanding is right, perhaps supporting the ServiceExport API (instead of the skupper annotation on a service) would be straightforward?

Any/all thoughts on this are welcome, I'm probably misunderstanding something fundamental...

Thank you!

grs commented 3 years ago

Hi! Thanks for taking time to look at skupper and give this feedback!

We could support the ServiceExport CRD as an alternative to an annotation on the service (though the preferred option is to annotate the deployment/statefulset/daemonset that is the target of the service).

As I understand KEP-1645, the expectation is that for every ServiceExport, there will be a ServiceImport created in all the matching namespaces. The ServiceImport CRD design seems to imply a layer 3 solution.

Also, as you note, skupper intentionally does not follow the 'namespace sameness' assumption and the solution envisaged seems to be a cluster level one (i.e. the same mechanism is assumed for the cluster as a whole, and indeed all those connected) whereas skupper does not require cluster privileges for flexibility.

So I don't think we could be properly 'compliant' with the KEP as specified, and my concern in just using one of the CRDs defined there in a slightly different manner might result in mismatched expectations.

I do recognise the value of having an existing CRD that we could use (particularly useful as a way of indicating status). I am just not sure yet whether it would on balance be more problematic to try and use this one in a manner it is not quite intended for.

What do you think?

rosenhouse commented 3 years ago

Yes, I think I agree on almost everything....

A couple thoughts:

imply a layer 3 solution

I think L4 would work? I haven't played with Skupper much yet, but if I'm reading the docs right, Skupper's support for tunneling raw TCP on a per-port basis should suffice here. For example, if the skupper instances were the ones creating the ServiceImport resources and associated EndpointSlices, they could ensure that the EndpointSlices pointed at the Skupper router's ports, basically the same way it works today with selectorless Services? (or is my mental model wrong?)

Specifically regarding

So I don't think we could be properly 'compliant' with the KEP as specified, and my concern in just using one of the CRDs defined there in a slightly different manner might result in mismatched expectations.

Yes, I have the same concern. I may bring this up with the KEP authors and see their thoughts -- the APIs feel useful even with Skupper's less-demanding trust model.

I'll reply here once I get more clarity!

grs commented 3 years ago

Yes, sorry, I think you are right. The IPs listed in ServiceImport could just be that of the skupper router. I'll try and find some time to play around with this idea and get a better feel for it.

rosenhouse commented 3 years ago

An update on

properly 'compliant' with the KEP as specified

I raised this topic at the May 11 SIG-Multicluster meeting and was encouraged to PR the KEP, so I did here: https://github.com/kubernetes/enhancements/pull/2734