unbounce / iidy

iidy (Is it done yet?) -- CloudFormation with Confidence
MIT License
52 stars 7 forks source link

Can't use preprocessor tags on "global" sections #213

Closed dannosaur closed 4 years ago

dannosaur commented 5 years ago

I'm building a series of stacks with common components, and to save rewriting almost the same large templates with minimal changes, I considered splitting each component's configuration/templates into separate files, and using !$merge to bring them all together as/when needed on a per-stack basis.

Consider this extremely cut-down example to illustrate what I'm trying to achieve here;

file1.yml

Resources:
  MemcachedCluster:
    Type: AWS::ElastiCache::CacheCluster
    Properties:
      Engine: memcached

file2.yml

  DatabaseInstance:
    Type: AWS::RDS::DBInstance
    Properties:
      Engine: postgres

stack.yml

$imports:
  memcached: ./file1.yml
  database: ./file2.yml

$defs:
  CustomResources:
    VPC:
      Type: AWS::EC2::VPC
      Properties:
        CidrBlock: 172.30.0.0/16

So if stack.yml contains the following (to illustrate the pre-processor is working and configured correctly);;

merged: !$merge
  - !$ CustomResources
  - !$ database.Resources
  - !$ memcached.Resources

The output becomes;

$ iidy render stack.yml
merged:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 172.30.0.0/16
  PublicSubnetA:
    Type: 'AWS::EC2::Subnet'
    Properties:
      VpcId: !Ref VPC
  DatabaseInstance:
    Type: 'AWS::RDS::DBInstance'
    Properties:
      Engine: postgres
  MemcachedCluster:
    Type: 'AWS::ElastiCache::CacheCluster'
    Properties:
      Engine: memcached

However, if the section merged is replaced with Resources;

Resources: !$merge
  - !$ CustomResources
  - !$ database.Resources
  - !$ memcached.Resources

then the following error is observed;

$ iidy render stack.yml
error Invalid resource type: undefined at Root.Resources: [
 {
  "_data": "OtherResources"
 },
 {
  "_data": "database.Resources"
 },
 {
  "_data": "memcached.Resources"
 }
]  

Is there any way to run this kind of merge/preprocessor logic on the top-level "Global" sections to allow this kind of configuration to work and render correctly?

tavisrudd commented 5 years ago

It looks like there's a bug there in the handling of Resources, but here's a work around: in your first attempt replace merged with $merge or $mergeIntoDocRoot (arbitrary suffix after $merge).

tavisrudd commented 5 years ago

There's also an undocumented feature I think you'd love. Email me at tavis at unbounce.com and I'll send you an example of it in use.

tavisrudd commented 5 years ago

Another option:

Resources: 
  $merge_custom: !$ CustomResources
  $merge_db: !$ database.Resources
  $merge_memc: !$ memcached.Resources
dannosaur commented 5 years ago

Ah, yes.. the latter option seems to do what I'm looking for.

I'll email you separately.

tavisrudd commented 4 years ago

Did #224 fix this?

dannosaur commented 4 years ago

I'll run some quick tests on master with some test templates I have hanging around from previous issues.

dannosaur commented 4 years ago

Turns out it didn't.