monken / cfn-include

Preprocessor for CloudFormation templates with support for loops and flexible include statements
MIT License
86 stars 10 forks source link

"real" context parameters? #36

Closed tolnai closed 2 months ago

tolnai commented 6 years ago

This is more of a question. I'm trying to come up with some combination of the available native CloudFormation and cfn-include tools to achieve the following - but so far without success:

So for example, here is the following:

Version: '2012-10-17'
Statement:
  Effect: Allow
  Action: 'sts:AssumeRole'
  Resource: 'arn:aws:iam::${AccountId}:role/${RoleName}'

(syntax simplified to avoid confusing Fn:.:Join structure) While the AccountId can be a global, simple parameter, I'd like to create separate AssumeRole policies for different target roles, but I don't want to create a separate source file for them. Fn::Sub can only substitute in strings, so as far as I see, when using !Include, there is no way to pass in "local constants". Am I right? Could this be done somehow, or if I want this level of IaaC, shall I invest into something else like Terraform?

Thanks!

tolnai commented 6 years ago

I figured out a way to do this with Fn::Map. We can create "context variables". Parent file:

Fn::Merge:
  Fn::Map:
    - [ 'Value' ]
    - VarName
    - !Include { location: OtherFile.yaml }

Included file:

Property: 'Value: ${VarName}'

But this is a bit cumbersome. For 2 variables I would need 2 Fn::Map, an Fn::Flatten, and an Fn::Merge. Since this is mostly useful with including parameterized files, we could use a "context" within the !Include statement (similarly as it was used for literals, which is now deprecated), or just create a new Fn::Context function, which is a shortcut for the map+flatten+merge syntax. What do you think?

monken commented 6 years ago

I think Fn::Map is your best bet for now but I understand the limitations and the overhead of writing these constructs. I'll look into providing something like Fn::Context or similar (reviving the context parameter to Fn::Include)

Megamiun commented 3 years ago

Hello, @monken, is there any plans of doing that?

I would like to use your lib to flatten/unroll a Cloudformation structure which use nested stacks, but at the current format, it isn't viable.

EDIT: If you had any idea of how to implement this but doesn't have time, could you send me the ideas, so I can implement and create an PR?

monken commented 3 years ago

Hey, could you give an example of how this would help? I do the following to merge multiple templates:

Fn::Merge:
  - tpl1.yml
  - tpl2.yml
Megamiun commented 3 years ago

On my case I have this generic stack for example:

---
AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  InboundSg:
    Type: String
  OutboundSg:
    Type: String
  Port:
    Type: String

Resources:
  Ingress:
    Type: AWS::EC2::SecurityGroupIngress
    Properties: 
      SourceSecurityGroupId: !Ref OutboundSg
      FromPort: !Ref Port
      ToPort: !Ref Port
      GroupId: !Ref InboundSg
      IpProtocol: tcp

  Egress:
    Type: AWS::EC2::SecurityGroupEgress
    Properties: 
      DestinationSecurityGroupId: !Ref InboundSg
      FromPort: !Ref Port
      ToPort: !Ref Port
      GroupId: !Ref OutboundSg
      IpProtocol: tcp

As you can see, there are three args that go in creating this link. If I have to manage multiple of them, there not only will be a conflict of names of the multiple Ingress and Egress, but it will be harder to manage them than passing a context to the map, even more if there are multiple levels of !Include

nmccready commented 2 months ago

This is possible now with Fn::Include.inject https://github.com/monken/cfn-include?tab=readme-ov-file#include-inject-state

nmccready commented 2 months ago

We need to cut a new release off master.