saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.1k stars 5.47k forks source link

Top SLS compilation does not behave the same as Docs describe #12483

Closed driskell closed 9 years ago

driskell commented 10 years ago

Hi

(This is taken from my comments on closed issue #5440 - @basepi recommended a new issue so here it is! Thanks.)

I have 4 environments, "base", "qa", "dev", "master", in that order defined in file_roots. These are branches in a git and I have my own deployment of this to the salt server (I do not use gitfs).

Each has a version of the top.sls file containing all environments (as you'd expect since it same file, just different revisions different branches)

The documentation describes that if an environment has info in top.sls in "base", it will always override all other top.sls. This is not the case - and checking the code there is no distinction between "base" and any other name, and no specific decision on ordering (and it seems alphabetical) - so in my case, even though in the top.sls and file_roots, "qa" is second down, because it is last alphabetically, that overrides all other top.sls files in all other environments.

Based on the docs, it seems to give the idea that "base" is a special name and is always authoritative. It also explicitly says that if an environment is defined in multiple top.sls files - the definition in the top.sls for that environment will be preferred. But the code seems to say otherwise.

The code seems to simply merge and overwrite, in alphabetical order. Which contradicts entirely the documentation.

For reference, from http://docs.saltstack.com/en/latest/ref/states/top.html:

The following is what lead me to think "base" was a special name (it doesn't say the FIRST environment wins, it says the "base" environment wins.)

21.24.15.3. HOW TOP FILES ARE COMPILED ... 1 . The base environment's top file is processed first. Any environment which is defined in the base top.sls as well as another environment's top file, will use the instance of the environment configured in base and ignore all other instances. In other words, the base top file is authoritative when defining environments.

The following is the bit saying a top.sls in an environment is authoritative for that environment if it contains config for it:

3 . For environments other than base, the top file in a given environment will be checked for a section matching the environment's name. If one is found, then it is used. Otherwise, the remaining (non-base) environments will be checked in alphabetical order. In the below example, the qa section in /srv/salt/dev/top.sls will be ignored, but if /srv/salt/qa/top.sls were cleared or removed, then the states configured for the qa environment in /srv/salt/dev/top.sls will be applied.

Although if I'm honest, point 3 does seem to contradict point 2 (below) - I would've expected point 2 to at least make an exception for point 3 - or point 3 come first

2 . If, for some reason, the base environment is not configured in the base environment's top file, then the other environments will be checked in alphabetical order. The first top file found to contain a section for the base environment wins, and the other top files' base sections are ignored. So, provided there is no base section in the base top file, with the below two top files the dev environment would win out, and the common.centos SLS would not be applied to CentOS hosts.

@basepi:

@driskell Could I convince you to open a new issue and insert all of this awesome information and detective work into that new issue? I think what we will want to do is bring the code in line with the documentation (I still haven't tested this myself, so I haven't verified that I see the same thing you do) if possible. I think the documentation's way is a good way, but we definitely need to get them in line with each other one way or another.

I agree the documentation is (mostly) a good way, and the behaviour should match. Assuming I haven't read the doc wrong though, and "base" is meant to be special and authoritative and overriding, then I think that should be configurable so I can call it something else, so maybe a "main_env: base" option with that as default.

Thanks guys!

cachedout commented 9 years ago

OK, I just about have a PR ready to go. Here are the changes I propose making:

After reviewing this in more detail, I think @driskell had the right idea in his earlier proposal. Therefore, I've done a few things:

1) Configuration files are now rendered as ordered dictionaries. I have added a new configuration file option called ordered_top_files as suggested by @driskell . It defaults to False, but when it is turned on, it will process environments in the order they are written in the configuration file.

2) I have added a configuration option called top_file_merging_strategy. This can be set to either merge or same (currently). If set to merge, the current behaviour is preserved. However, this can be influenced by the ordered_top_files setting above, in what should hopefully be an obvious way. If it's not, please let me know. :]

I did not implement fallback options because I feel like environments are already very difficult to reason about and I feel like adding fallback behaviour will only increase that complexity. Differing opinions are welcome of course. :]

3) I created a default_top option. If this is set, and no environment is speficially requested (via configuration or via a saltenv argument) and the same value is applied to the top_file_merging_strategy option, this is the environment that will be used.

Thoughts? I'll work on cleaning this up and submitting a PR, probably tomorrow.

mathom commented 9 years ago

@cachedout I'd be very interested in seeing your PR. Being able to disable the topfile merging would greatly simplify our environment setup and test process.

cachedout commented 9 years ago

@mathom I was just waiting for any feedback. :] I am planning on getting this cleaned up and submitted today. Thanks for being willing to review it. Much appreciated.

jmelfi commented 9 years ago

@cachedout I will love to test this.

I assume that top_file_merging_strategy being set to merge by default and ordered_top_files being set to True would pull them from the top down.

Using this method, we would need to define the most generic first and then the most custom at the bottom to get a proper merge overwrite or is this resolved otherwise with the change to using ordered dictionaries? I'm interested in seeing how much would need to be altered to use different branches in our git repositories to deploy our code base from a single top file for definition.

Thanks for all the work put in as this is a large problem that is quite complex from what I can understand.

cachedout commented 9 years ago

@jmelfi I ended up running into a problem with ordered_top_files and so I'm changing it to support an order: flag for each env in file_roots. In the end, I think this is better since depending on ordering in a YAML dictionary is just asking for trouble. To answer your question though, yes, with the order flag set in your envs, the behaviour you describe is what should happen.

cachedout commented 9 years ago

OK @jmelfi and @mathom. Have a look at #24878 and let me know what you think.

jmelfi commented 9 years ago

@cachedout I'll give it a review over the next hour or so. Thanks for the update!

cachedout commented 9 years ago

@jmelfi Super. Be sure to configure things as described in the PR. It won't (or shouldn't) change behaviour out of the box. I'll be here if you have questions. Thanks for the testing help!

DanyC97 commented 9 years ago

wow, this was a long issue to read :)) Would love to see this merged soon, thx for the effort!

cachedout commented 9 years ago

I have merged this fix into develop and it should be backported into the 2015.8 branch in the next day or two.

basepi commented 9 years ago

:+1:

rallytime commented 9 years ago

Back-port is in #26214. :)

jmelfi commented 9 years ago

@cachedout sorry for a late reply here. I was able to get the sorting to work properly after using the PR work. Thanks you guys for this hard work. I've been working the last week on moving from our puppet code completely into saltstack.

You guys are awesome!

cachedout commented 9 years ago

@jmelfi I'm so glad! Thanks for testing this. For the time being, I am going to close this issue since we're using it to track progress toward our release. That doesn't mean, however, that people can't continue to post feedback here. We'll re-open it if we need to. Thanks again!