hamlet-io / engine

Framework for managing cloud infrastructure via templates. It is part of the broader Hamlet devops framework.
GNU General Public License v3.0
5 stars 5 forks source link

feat: private endpoint services #2106

Closed ml019 closed 1 year ago

ml019 commented 1 year ago

Intent of Change

Description

Extend the processing of endpoint services to support privately defined endpoints, as opposed to the public ones offered by cloud providers.

These endpoints are assumed to be required in all zones supported by the network of the tier in which the gateway is defined.

Motivation and Context

Support the use of customer defined service endpoints in AWS.

How Has This Been Tested?

Local template generation. I'll also apply on customer site.

Related Changes

Prerequisite PRs:

Dependent PRs:

Consumer Actions:

ml019 commented 1 year ago

Not sure if this is the best way to handle this, or whether I should instead use links from a gateway destination to an external service component which has the serviceName as an attribute. Maybe both make sense - thoughts welcome.

roleyfoley commented 1 year ago

You can include private ones in the current implementation.

Under the region config

    "Regions": {
        "ap-southeast-2": {
            "Locality": "Sydney",
            "Zones": {
                "a": {
                    "NetworkEndpoints": [
                        {
                            "Type": "Interface",
                            "ServiceName": "com.amazonaws.vpce.ap-southeast-2.vpce-svc-abc123"
                        }

Then under the network endpoint groups

    "NetworkEndpointGroups": {
        "privateService": {
            "Services": [
                "com.amazonaws.vpce.ap-southeast-2.vpce-svc-abc123"
            ]
        },

Then when creating the gateway endpoint you can specifiy privateService like you would for the storage groups etc. The only pain though is that sinec the Network Endpoints is currently an array you need to redefine the ones that you already had

ml019 commented 1 year ago

Yeah - I didn't want to fiddle with the regional data as that naturally comes from the masterdata for the cloud provider. I looked at doing it the way you suggested initially, but the arrays and the overriding of provider data felt wrong,.

The other thing I wanted to avoid what having to do it for every zone in the regional data so basically always using every zone enabled by the solution. For VPC endpoints associated with customer defined service I think that will generally be the case, and if not, then the fallback is the way you suggest.

ml019 commented 1 year ago

@roleyfoley are we good to merge this or do you still have concerns? I'm at the point where I want to test it with the customer.

ml019 commented 1 year ago

BTW I suspect we may have wait issues with customer endpoint services as connections need to be accepted. I want to get to the point of being able to try one to work out next steps.

There is an auto-accept model based on explicit provision of connecting principles whihc I think makes more sense but will need to persuade the CE folks of its merits.