sparkleformation / sparkle_formation

Ruby orchestration templating library built with unicorns and rainbows
http://www.sparkleformation.io
Apache License 2.0
222 stars 47 forks source link

Naming collision between dynamics and templates #235

Open jeffb-stell opened 7 years ago

jeffb-stell commented 7 years ago

This one's a bit weird, and I don't have everything needed to replicate the issue.

A sparkleformation template with the same filename as a dynamic CAN result in the dynamic file failing to load.

Conditions:

I discovered that changing a line in sparkle.rb's load_parts! fixed the problem: Dir.glob(File.join(root, '**', '**', '*.{json,rb}')).each do |file| becomes Dir.glob(File.join(root, '**', '**', '*.{json,rb}')).sort.each do |file|

and fixes the issue (apparently). I suspect the issue is the non-deterministic way Dir.glob returns directory entries. In my case, sorting changed the entries from

"/var/lib/jenkins/workspace/sid-ecs/provisioning/ecs_cluster.rb", "/var/lib/jenkins/workspace/sid-ecs/provisioning/dynamics/ecs_cluster.rb", "/var/lib/jenkins/workspace/sid-ecs/provisioning/dynamics/ecr_repository.rb", "/var/lib/jenkins/workspace/sid-ecs/provisioning/prod_ecr_repository.rb"

to

"/var/lib/jenkins/workspace/sid-ecs/provisioning/dynamics/ecr_repository.rb", "/var/lib/jenkins/workspace/sid-ecs/provisioning/dynamics/ecs_cluster.rb", "/var/lib/jenkins/workspace/sid-ecs/provisioning/ecs_cluster.rb", "/var/lib/jenkins/workspace/sid-ecs/provisioning/prod_ecr_repository.rb"

That is, load_parts! evaluating dynamics before templates fixed my problem. I don't think the sort I added is enough to enforce deepest-file first, and in fact I went back and just renamed files.

jeffb-stell commented 7 years ago

stack trace gist: https://gist.github.com/jeffb-stell/bbe564c5be38da0a81303a5396db4145

chrisroberts commented 6 years ago

I'm working on getting a local repro setup and I think a bigger part of it is setting up the custom way you are invoking the template compilation (since you are not using sfn). The glob listings you show in the linked gist show the missing dynamic only being loaded once in the three different examples, which leads me to believe that it may be an issue with how things are being configured prior to the actual template compilation. Is that something you can share? It doesn't need to be the entire file, just the initial setup up to the compile call.