search

aws / aws-application-networking-k8s

A Kubernetes controller for Amazon VPC Lattice
https://www.gateway-api-controller.eks.aws.dev/
Apache License 2.0
162 stars 47 forks source link

Attaching a Service Network to a VPC different that the one where the EKS cluster is #265

Open nicoaws opened 1 year ago

nicoaws commented 1 year ago

As a customer I'd like the option to specify a list of VPCs different to the one the EKS cluster is deployed in. I suggest modifying the current annotation to: application-networking.k8s.aws/lattice-self-vpc-association: "true" and add another application-networking.k8s.aws/lattice-vpc-association: "[vpc-1-id],[vpc-2-id]"

zijun726911 commented 1 year ago

Thanks for sending an issue to the aws-application-networking-k8s! we will consider your suggestion in our road map.

As a customer I'd like the option to specify a list of VPCs different to the one the EKS cluster is deployed in.

Would you mind to provide more detail and backgroud for this use case?

Are you expecting the following behaviour? say, current EKS cluster1 which use vpc-1-id if you apply following config:

   apiVersion: gateway.networking.k8s.io/v1beta1
   kind: Gateway
   metadata:
     name: my-hotel
     annotations:
        application-networking.k8s.aws/lattice-self-vpc-association: "true"
        application-networking.k8s.aws/lattice-vpc-association: "[vpc-1-id],[vpc-2-id]"

The controller in the backend will create vpc-1-id to this serviceNetwork(gateway) association in the vpcLattice

and then, when you switch to another EKS cluster, says, cluster3 that using vpc-3-id if you apply the same config:

   apiVersion: gateway.networking.k8s.io/v1beta1
   kind: Gateway
   metadata:
     name: my-hotel
     annotations:
        application-networking.k8s.aws/lattice-self-vpc-association: "true"
        application-networking.k8s.aws/lattice-vpc-association: "[vpc-1-id],[vpc-2-id]"

The controller will NOT do vpc-3-id to servicenetwork association(same serviceNetwork but a new k8s gateway in cluster3) to current cluster3's vpc?

nicoaws commented 1 year ago

In the first case, yes the gateway will associate the new (or existing) service network to the VPC in the list. If another gateway is created in the same EKS cluster (or elsewhere), self-vpc association works as usual, but association errors out if any VPC in the list is already associated to a SN.

We need to think about transactional integrity. If any of the components fails, do we fail the creation of the gateway resource, or just output an error in the controller logs?

liwenwu-amazon commented 1 year ago

We recommend customer following this architecture, where config cluster is used to configure gateway(s) and HTTPRoute(s) and data cluster decides which gateway(s) to associate its VPC to.

nicoaws commented 1 year ago

My use case is different. I only have one cluster where a service is implemented and have N other VPCs that need to consume the service. Therefore I’d like to be able to associate a service network created with a gateway object in kubernetes to an arbitrary list of VPCs.

liwenwu-amazon commented 1 year ago

If those VPCs belong to a K8S cluster, they can have a gateway object defined in their cluster and controller in their cluster will associate their VPC to the service network.

nicoaws commented 1 year ago

There is no K8s cluster in those VPCs.

On Fri, Jul 14, 2023 at 12:58 AM liwenwu-amazon @.***> wrote:

If those VPCs belong to a K8S cluster, they can have a gateway object defined in their cluster and controller in their cluster will associate their VPC to the service network.

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-application-networking-k8s/issues/265#issuecomment-1635069659, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLRAKK7M7WFUCRS3MUT76TXQCDTXANCNFSM6AAAAAAY5OFXTA . You are receiving this because you authored the thread.Message ID: @.***>

liwenwu-amazon commented 1 year ago

For those VPCs which does NOT have any Kubernetes cluster, you can use lattice CLI or console to associate them to the service network required.

nicoaws commented 1 year ago

If I want to associate a VPC to a SN I create within k8s I need to have a way to programmatically retrieve the ID of the SN from Terraform.

Currently the AWS provider doesn’t have a data resource for a SN that is capable of filtering by tags or any other mechanism.

My understanding is that this is because the API action doesn’t implement a filter.

On Fri, 14 Jul 2023 at 18:44, liwenwu-amazon @.***> wrote:

For those VPCs which does NOT have any Kubernetes cluster, you can use lattice CLI or console to associate them to the service network required.

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-application-networking-k8s/issues/265#issuecomment-1636182903, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLRAKJ7J3LGVKDNO3VAYTTXQGAQNANCNFSM6AAAAAAY5OFXTA . You are receiving this because you authored the thread.Message ID: @.***>