[FR] aws cli compatible render

[lorienx0]

Is your feature request related to a problem? Please describe. It's not very convenient to rely on additional dependencies (like ecs_composex and its own dependencies) when setting up CI/CD pipeline for updating a stack. But trying to pass rendered stacks into aws cli w/o additional post processing seems impossible.

Describe the solution you'd like Add a command line parameter to ecs compose x cli tool, that instructs it to render aws cli ready templates. It could target deploy or stack-create/stack-update workflows or both. Render could use the bucket name parameter and a new bucket key parameter to render the proper TemplateUrl fields.

Additional context It would be nice to just pull a generic aws-cli docker image and feed it the output of ecs_composex that is a part of a project. But current output format of render command renders TemplateUrl fields that point to local files. So they cannot be just uploaded to s3 and used with create-stack or update-stack commands. It's also incompatible with the deploy command - it doesn't seem to work with multiple template and parameter files.

[JohnPreston]

Hello @lorienx0 You are probably looking for the ecs-compose-x create which does the same as render but uploads the certificates to the given bucket and the root template etc. have the TemplateUrl already in it. That's how we do all our deployments via CICD pipeline. Let me know if that is what you were looking for. If you ran ecs-compose-x init it will create a default bucket for you otherwise set the bucket from the CLI

[lorienx0]

This is also how I use the tool currently. The issue is that this workflow brings back the dependency on ecs_composex. I'd really appreciate an option to use ecs_composex as a development tool to render templates into the project, which then can be used with bare aws cli. Render accepts a bucket name argument, but it seems like it's never used. Also, maybe I'm wrong here, but refs to local files in templates seem to have no use in general ? As neither aws cli nor ecs_composex cli can do anything useful with them.

[JohnPreston]

I am not entirely sure what the FR then is :thinking: Installing ecs-composex (and its dependencies) allows to render the templates at anytime. If you want to use a docker image instead, there is one. If you want to create the templates within the project and version them within GIT, I don't see a reason not to do that.

When using render, the path to local files is "useless", indeed, but the reason why it's there is, the CFN Stack (nested) requires the property to be set for it.

Happy to have a live catchup to understand better what you want out of this FR :smiley:

[lorienx0]

In other words, the request is to make render act like the create command when we provide it with a bucket name. So it would render s3 TemplateUrl fields, but wouldn't upload to s3. The purpose is a cleaner workflow that would not require manual edits or automated post processing for the rendered content. The logic for this should already be there because create exists, and this would be basically create, but w/o the upload step.

[JohnPreston]

Ahhh. So something like ecs-compose-x create --no-upload. Which is fine, however, I'd question why you want the path in the TemplateUrl if that file does not exist :thinking:

Atm the path to the files in S3 has a date + random bit in order to allow for no overlap if you run the same pipeline within the same time.

So I would imagine that in addition to --no-upload one would want a --s3-prefix so that you know the path to these files is always correct.

I am happy to do a --no-upload to create but I am unsure of the purpose as the TemplateUrl has to be a valid path to a file that does already exist.

[lorienx0]

Upload to s3 to the right location (so that TemplateUrl point to correct nested templates) would be on the user, as they explicitly opt in. The dynamic part of the s3 uri could either be overridden with an additional argument, or just omitted entirely.

I'd personally prefer this to be implemented as a part of the render command, because it already has a --bucket-name argument that does nothing. But it isn't important ofc. I'd be happy either way.

[JohnPreston]

Sounds good, I will see about a render --s3-prefix which will allow you to set which logical folder in the bucket the files would otherwise be found, and if not set, the root of the bucket

[JohnPreston]

@lorienx0 will look at working on the feature, and, well, my backlog, this weekend :smile:

[lorienx0]

Thank you, much appreciated !

[JohnPreston]

@lorienx0 try installing from the main branch, you should have the render --s3-prefix argument available for you to test with this.

[lorienx0]

@lorienx0 try installing from the main branch, you should have the render --s3-prefix argument available for you to test with this.

It worked as expected, thank you 🎉