Support for multi-region deployments

[bisoldi]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request I want the region (or regions) of deployment to be a property of the environment. More simply, I want to be able to use Proton to construct "environment" stacks across multiple regions and deploy a multi-region (active/active) processing pipeline.

Not sure if this is even feasible for Proton to abstract away from me and take care of itself as it's a region-specific service, so this may be more valuable as a request for documentation/guidance.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? I've designed and roughly architected/provisioned a multi-region pipeline that's segmented into 4 layers:

1. A dual region scheduler/kick-off layer that only operates in 1 region at any one time and would respond to health issues in the primary region by disabling the triggers in the primary region and enabling them in the secondary region. This layer has things like multi-region RDS, global Dynamo tables, etc. and kicks off operations in the next layer. (Active/Passive)

2. A many/all region data ingestion layer triggered by the previous layer and that acquires (either pulls or receives) the data to be processed. Active/Active.

3. A dual region processing/enrichment layer that operates in the same 2 regions as the scheduler layer, but is active/active.

4. A dual region persistence layer with predominately shared components that operates in the same 2 regions as the processing layer and is also active/active.

Some notes on the above:

• Each layer is further segmented into shared and service specific resources. The individual services "plug-in" to the architecture by installing service-specific Rule's into a region and environment specific EventBridge.

• The infrastructure in each region is deployed with 2 different Cloudformation templates; a region-wide template which provides the RestApi (entry point into the layer), a POST Method and an IAM Role, and a stage-specific template which provides the EventBridge, Route53, Domain Name, ApiGateway::Deployment, etc.

• Thankfully, each layer is deployed using the same 2 Cloudformation templates (region-wide and stage-specific).

• Each layer communicates with the next one via regional API Gateway's with a custom domain name and round-robin Route53.

• Everything is done in Cloudformation and I have fairly rudimentary Python scripts that deploy/update and persist the various "touch points" to a file that then each service's deployment script reads, but...I don't really have a good/easy way to manage all of this automatically.

If Proton can't natively support multiple regions in one "Environment", how do you recommend we go about managing a multi-region architecture like the one I outlined above??

Are you currently working around this issue? Manually, though it's not in production...still early stages of development/testing.

[rafavallina]

@bisoldi took me a bit to come back to this, apologies! Three pieces to this:

1. We indeed would like region to be part of the definition of an environment, so indeed you could have a multi-region process like this.
2. Your architecture hints at another feature that we've discussed internally but haven't fleshed out enough to put in the roadmap: nested environments. In your case, your region-wide templates would be "base" environments, and your "layer specific" templates would be nested on those. That allows the nested environment to share resources between them
3. This said, the process of managing the deployment of services through this environments is not a planned feature of Proton at this time - rather, we rely on the pipeline to be in charge of deploying code and calling Proton in every stage to deploy as needed. So in your case, you can rely on Proton to oversee environments and services on them (and, if using nested environments, have the appropriate common resources and outputs available), but you will still need something like your Python script to manage the deployments as they progress through your pipeline

I'm keeping this issue open as the title accurately represents a feature that we want to deliver

[viniciuschiele]

@rafavallina any progress on multi-region deployments?

I thought by connecting Proton to another AWS account I would be able to have my environment in another region. Still, it looks like the Management Account and the Connected Account can only use the same region.

I was trying to create two production environments prod-eu and prod-us, and use the same pipeline to deploy a service to both environments.

[haakond]

@rafavallina any progress on multi-region deployments?

I thought by connecting Proton to another AWS account I would be able to have my environment in another region. Still, it looks like the Management Account and the Connected Account can only use the same region.

I was trying to create two production environments prod-eu and prod-us, and use the same pipeline to deploy a service to both environments.

+1

[rafavallina]

@viniciuschiele @haakond we still have this in our roadmap, but unfortunately I don't have a specific date for delivery. Some customers have managed to work around this issue by using CloudFormation stack sets, but it can take significant work to set up.

Should we schedule time to discuss the use case in person? Getting more details will help us prioritize.

[ronan-cunningham]

Hi @rafavallina - is it possible to get an idea where/how stacksets were used in the workarounds you refer to?

We (https://axial3d.com/) have a multi-account code-pipeline that deploys into multiple regions in parallel. It's a SAAS product with APIs in different regions for data residency purposes.

I love the idea of the environment and service definitions in Proton, and I'm keen to test it out. Production for us though is one account and multiple regions, all running the same versions of code. I'm used to our pipeline building the code into a single docker image and deploying to multiple regions ( in parallel ) in multiple accounts (serially).

It seems to use Proton we would need to:

• represent each environment account in every region
• create multiple service instances in each region That's a lot of repetition. It's not clear to me where an AWS Proton Managed deployment pipeline would fit in for our use case-I'm struggling conceptually with that though could possibly/probably create a non-proton managed one fairly easily.

[nundys]

+1 Another use case: We have been waiting to deploy in a Proton non-supported region (Mumbai) for many months now and will have to do non-ideal CF stacksets instead.