aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.51k stars 3.85k forks source link

Make it easy to turn ecs TaskDefinitions into json strings #8304

Open tjtaill opened 4 years ago

tjtaill commented 4 years ago

in the package @aws-cdk/aws-ecs on the class TaskDefinition please add a toJSON method that returns the complete task definition as a JSON

Use Case

This would be used to have a template taskdef.json for codedeploy.

Proposed Solution

Add toJSON method on TaskDefinition

Other


This is a :rocket: Feature Request

theGreatHeisenberg commented 3 years ago

In unit tests, is there a way to validate if appropriate environment variables are set? Currently, I am not finding any way to achieve this. If there's a way to turn taskDefinitions into JSON, that would allow me to validate environment variables.

madeline-k commented 1 year ago

Hey @tjtaill, can you provide more detail on your use-case? Once you have the TaskDefinition in json-form, what would you do with it? Pass to a CodeDeploy construct? I am asking because it might be better to provide a specific integration with the CodeDeploy module, or a more specific API for this than just toJSON. Also, there are tokens and references to consider, so it may not always be possible to render the TaskDefinition as usable json, until synthesis is complete.

@theGreatHeisenberg , for unit testing your CDK applications, you can make assertions on anything that is in the cloudformation template. The environment variables will be there. See the assertions library: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.assertions-readme.html

madeline-k commented 1 year ago

Also, this issue has not gotten any attention in a long time, so we may close this if there is no further engagement.

DinosaurDad commented 1 year ago

Once you have the TaskDefinition in json-form, what would you do with it? Pass to a CodeDeploy construct?

@madeline-k : I am new to CDK and have put together a blue/green deployment for containers on ECS instances. What you mentioned is exactly something I've been looking for.

  1. We create an Ec2Service, which requires an Ec2TaskDefinition, which requires a ContainerDefinition.
  2. We create a Pipeline with a CodeDeployEcsDeployAction, which requires separate taskdef.json and appspec.yaml files.

The Pipeline needs artifacts that would be much, much cleaner if they could be created directly in the construct. Ideally taskdef.json would be unnecessary, with the Deploy task definition being provided by the Ec2TaskDefinition construct. Bonus points for something to have the appspec.yaml also created by the construct.

An alternative would be that the Ec2Service doesn't require the task definition, because the pipeline seems to immediately deploy a task and container that replace what the construct kicked off.

Maybe I'm overlooking something, but I've poured through a lot of documentation and the AWS console and can't find a direct way to wire these without redundancy and likely something getting out of sync.

tmayr commented 1 year ago

Once you have the TaskDefinition in json-form, what would you do with it? Pass to a CodeDeploy construct?

@madeline-k : I am new to CDK and have put together a blue/green deployment for containers on ECS instances. What you mentioned is exactly something I've been looking for.

  1. We create an Ec2Service, which requires an Ec2TaskDefinition, which requires a ContainerDefinition.
  2. We create a Pipeline with a CodeDeployEcsDeployAction, which requires separate taskdef.json and appspec.yaml files.

The Pipeline needs artifacts that would be much, much cleaner if they could be created directly in the construct. Ideally taskdef.json would be unnecessary, with the Deploy task definition being provided by the Ec2TaskDefinition construct. Bonus points for something to have the appspec.yaml also created by the construct.

An alternative would be that the Ec2Service doesn't require the task definition, because the pipeline seems to immediately deploy a task and container that replace what the construct kicked off.

Maybe I'm overlooking something, but I've poured through a lot of documentation and the AWS console and can't find a direct way to wire these without redundancy and likely something getting out of sync.

Right now I'm in exactly the same boat, and seems like most documentation either asks you go copy a big json from the AWS console or assumes you already have that .json file somewhere, which as you state, we don't because it's all set through CDK.

Curious on potential workarounds here or if we're clearly misunderstanding something.

tjtaill commented 1 year ago

Yeah sorry I haven't responded I found a good work around for this by using aws cli to fetch the task definition in json format, still think it would be convenient to get it from the object itself with a toJSON method but less critical now for me

tmayr commented 1 year ago

@madeline-k Is there any smarter way to pass it to a CodeDeploy construct other than fetching/maintaining a json plus the CDK code for the template definition?

omrta-dev commented 1 year ago

@madeline-k For nested stacks it should be possible since the taskDefinition from the parent stack should be fully rendered before the nested stack gets created?

Our workflow is to create an ecs cluster + services as a parent stack and then create the CodePipelines as a nested stack.

Parent:

Nested:

Because CodeDeploy requires a TaskDefinition file as well as an appSpecTemplate which we either need to manually source or attempt to generate ourselves at the moment it makes having a workflow of dynamically creating a CodePipeline basically unfeasible with the current lack of toJson or Artifact.fromObject or similar functionality.

I can attempt to implement this feature but would need guidance on what an expected workflow would look like, I propose something along the lines of:

  1. The ECS Task Definition UI can already generate a taskdef.json file, im assuming there is an API call to fetch that given a taskDefinition ARN.
  2. Upload the above taskdef.json as an artifact to s3 bucket XYZ
  3. Now that the taskdef.json is available as a .zip in s3, we can set the artifactBucket to the XYZ s3 bucket
  4. Artifacts containing the taskdef.json are now available for use in the CodePipeline.

EDIT: Actually would this be better as a higher level construct similar to a LoadBalancedEC2Service but with functionality to generate a pipeline? Or can it get added to a current construct?

DinosaurDad commented 1 year ago

As feared, we just got burned by our CodeDeploy task definition JSON being out of sync with our CDK props.

omrta-dev commented 1 year ago

Any updates or workarounds for this? It's been a month and no one from the cdk project has responded. There NEEDS to be a way to construct a CodePipeline as CDK code using the provided services. As mentioned above a nested stack should be able to resolve any tokens from the creation stack(the ecs cluster + the services). I believe there currently is no clean solution.

Luxiorawa commented 9 months ago

I also heavily need this feature

sergiowww commented 1 month ago

It would be great if we could use only taskDefinition on CodeDeployEcsDeployAction and get rid of appspec.yml and taskdef.json. That's too much redundancy.

github-actions[bot] commented 3 weeks ago

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.