GitOps management of service specifications

rafavallina commented 3 years ago

Store the specification details of a service in a YAML file in the service repository, so that developers can make changes to it by using git operations

mikesir87 commented 3 years ago

Just to make sure I understand the request here, are you indicating this would allow development teams to have a manifest (of some sort) that would be watched and deployed when changed? Just verifying this is the dev side of things and not the service template spec. I'm actually in favor of both, but think this is focusing on the dev side of things.

rafavallina commented 3 years ago

@mikesir87 You are 100% correct. We indeed would like to do both sides, but this particular issue is for devs.

JohnPreston commented 3 years ago

Why would AWS CloudFormation "Conflg" files in which are today in JSON and used by CodePipeline not work for that purpose? Is this to allow the same functionality but written in YAML instead of JSON?

rafavallina commented 3 years ago

The difference is in how developers interact with CloudFormation. Proton abstracts the CFN away - the expectation is that with a single config file you get your architecture and pipeline. This specification file would be the single template that developers use to define all pieces of their service, including the configuration of each instance and their pipelines. We'd expect administrators to be using the config files, not developers.

JohnPreston commented 3 years ago

The difference is in how developers interact with CloudFormation. Proton abstracts the CFN away - the expectation is that with a single config file you get your architecture and pipeline. This specification file would be the single template that developers use to define all pieces of their service, including the configuration of each instance and their pipelines. We'd expect administrators to be using the config files, not developers.

I probably need to get more into it than the couple things I looked at to get a better feel of it. Just to clarify, what are we calling an instance here when we say

including the configuration of each instance and their pipelines

Is that 1 service / 1 group of services ? the resulting Stack Proton will create ?

rafavallina commented 3 years ago

Good callout! Didn't explain myself well. In Proton, we have 3 core concepts:

Environments, which represent shared resources (e.g. an ECS cluster)
Services, which represent a workload running in multiple environments with its pipeline (e.g. a Fargate web service running in staging and production with its Jenkins pipeline)
Service instances, which represent the instantiation of a service in an environment (e.g. the staging instance of the aforementioned Fargate web service). And instance will often be a single CFN stack, but it could be more (if you create a Proton template that contains multiple CFN templates, for instance)

At the time that developers create services in Proton, they specify how many instances they want and in which environments they should be. And instances can be managed separately (e.g. you can deploy to staging and not deploy to production).

JohnPreston commented 3 years ago

Thanks for the clarification @rafavallina Makes perfect sense to me then, and I suppose "good news for me" is that we should be able to copy-paste a lot of what's been done at work and leverage my project to generate all the environments and services (templates effectively). Cheers :rocket:

kohidave commented 2 years ago

Just to make sure I understand the request here, are you indicating this would allow development teams to have a manifest (of some sort) that would be watched and deployed when changed? Just verifying this is the dev side of things and not the service template spec. I'm actually in favor of both, but think this is focusing on the dev side of things.

by the way - we did launch "Template Sync" which syncs templates from git :)

kohidave commented 2 years ago

Howdy folks - just wanted to poke this issue to let ya'll know we're knee deep in the implementation of being able to sync your service instance spec's directly from Git (without using GH Actions, CodePipeline, etc.).

kohidave commented 1 year ago

Howdy folks!!!

We've launched our initial release of this feature. Feel free to try it out and leave feedback! Really appreciate all ya'lls input on this!

https://aws.amazon.com/blogs/containers/announcing-git-based-service-deployments-with-service-sync-for-aws-proton/

aws / aws-proton-public-roadmap

GitOps management of service specifications #11