Referencing more than sceptre_user_data in jinja2 template

[et304383]

I'd like to reference global variables in config.yaml such as "environment".

I don't want to have to duplicate these values in every single stack config file.

The docs suggest that other vars should be accessible via

sceptre.environment.Environment().config

https://sceptre.cloudreach.com/latest/docs/environment_config.html

Not sure exactly where this can be used though.

Also, I'd like to be able to reference, say, project_code in a jinja2 template. Can't seem to do that either.

Can you please look into exposing all variables to jinja2 templates?

[et304383]

I have managed to work around this slightly by declaring a single variable in my stack config file that contains all the environment config values:

sceptre_user_data:
  environment_config: {{ environment_config }}

Then referencing by:

{{ sceptre_user_data.environment_config.project_code }}

[ngfgrant]

@et304383 Did you resolve your issue? Are you happy for me to close this?

[et304383]

I guess my main problem is that in every stack config file I have to explicitly pass in the environment config.

It would be nice if it was just available like this:

{{ environment_config.project_code }}

[ngfgrant]

Does https://github.com/cloudreach/sceptre/issues/256 describe the desired outcome?

[et304383]

I don't see how. I just want environment config to be available to all stacks without having to pass in to sceptre user data.

[mattvonrocketstein]

Issue #256 does seem kind of related, but the namespaces/global-config/command-line stuff it is mentioning seems to propose extra conventions. What makes the most sense to me for this ticket is just putting both environment_config and stack_config in the jinja context if possible, because this actually makes existing conventions from jinja more symmetric with python resolvers / hooks.

[et304383]

Making environment_config available in the jinja2 context would be the most flexible.

[canvaswang]

I agree. It will be a lot cleaner if we can simply have access to environment_config in jinja2 template without having to explicitly adding sceptre_user_data section in stack config file.

[ngfgrant]

This should be addressed in v2. Would anyone be willing to test on a simple setup?

[m1keil]

I don't believe this is. Just looking at the Stack template() method:

    @property
    def template(self):
        """
        Returns the CloudFormation Template used to create the Stack.

        :returns: The Stack's template.
        :rtype: str
        """
        self.connection_manager = ConnectionManager(
            self.region, self.profile, self.external_name
        )
        if self._template is None:
            self._template = Template(
                path=self.template_path,
                sceptre_user_data=self.sceptre_user_data,
                s3_details=self.s3_details,
                connection_manager=self.connection_manager
            )
        return self._template

I believe that in order to provide solution here, we'll need to pass stack_group_config as well as numerous other variables from the stack object like name, project_code, region etc.

[zaro0508]

I can confirm that @m1keil is correct, this issue has not been addressed in V2. Also {{ environment_config }} isn't available in V2 either so @et304383 workaround doesn't work in V2. My workaround for this issue is to create the sceptre-identity-resolver. This is still just a workaround, I agree with @et304383 comment: "I just want environment config to be available to all stacks without having to pass in to scepture user data".

[et304383]

Still not resolved in V2. How do I get the stack group name inside a stack group file?

I want to be able to do this in a Jinja2 template:

{{ sceptre_user_data.stack_group_name }}

[et304383]

Well, it seems that another magic variable is missing documentation, and I only discovered this by dumping stack_group_config:

stack_group_path

I searched the docs for this: https://sceptre.cloudreach.com/dev/search.html?q=stack_group_path&check_keywords=yes&area=default

Please document this. It's a critical value that some of us call "environment" and need to reference.

[mattvonrocketstein]

It's been a while since I thought about this but.. here is the relevant code, right? https://github.com/Sceptre/sceptre/blob/d45ffafcf9105376a81d4f75b538130f29890e33/sceptre/config/reader.py#L417 (this is master as of today, just a hash for a permalink)

Aren't we just saying here that after line 417 in the first link, we want to additionally add:

self.templating_vars.update(environment_config=stack_group_config)

FWIW, my org recently did the v1 -> v2 upgrade, but that only took us as far as 2.3.0 and not up to the bleeding edge. Unfortunately rather than returning to mainline, we still had to port a lot of stuff from our v1 fork to a v2 fork along the way. These are mostly minor things but several items in that category (and it's easier to fork to preserve backwards compatibility if you have hundreds of stacks you don't want to change quickly). So this issue, as well as the custom jinja filters issue i mentioned here, which was closed but afaik never resolved, is part of the reason we need to maintain our fork. Would be great to see those reasons get smaller over time.

[cleonard-godaddy]

bump? was there any resolution here?

[jfalkenstein]

Hi @cleonard-godaddy. The answer is "no but kind of."

No, nothing further has technically been added to the Jinja rendering. However, there is a new resolver which might help you out now: !stack_attr. With this resolver, you can reference other stack attributes (be they resolvers or plain old field values) and it will resolve to that value. Thus, while you can't directly access other attributes by default, you could do something like this:

sceptre_user_data:
    my_template_bucket: !stack_attr template_bucket
    stack_parameters: !stack_attr parameters
    stack_group_config: !stack_attr stack_group_config

Maybe this will help?

[alex-harvey-z3q]

@zaro0508 based on your comments above and Jon's I think we have a solution that has worked well enough for the current generation of Sceptre users and we should close this now. Thoughts?

[mattvonrocketstein]

we should close this now

The idea of "this environment name" or "this group name" remains a basic and critical value that's used in virtually every stack template that anyone will ever write. And it's still useful to provide this somehow to all templates pretty directly without making a lot of assumptions about what's inside the stack config and without requiring custom plugins, because that is what is least surprising and what people really expect to see by default.

If you require the work-around that

sceptre_user_data:
  stack_group_config: !stack_attr stack_group_config

or similar is present inside every config, this is effectively requiring tight-coupling between templates and configs, and thus lowers the reusability of templates. That undermines the whole goal of sceptre as I understand it, because why separate config/templates anyway if they always have to be changed together? Other issues include but are not limited to:

1. Repeatedly running into problems like env_name vs environment_name is frustrating, but that's what tends to happen if you have to always explicitly inject a variable that should be available by default.
2. If you can manage to establish stable naming conventions, then non-experts will forget that they are conventions, and then be confused when it doesn't work with a from-scratch stack that's created without copying an old one from existing files.

I could go on in more detail about how small things like this are surprisingly harmful for fostering widespread adoption, but maybe this is clear enough already.

If there's no acceptable simple fix similar to what's mentioned in https://github.com/Sceptre/sceptre/issues/355#issuecomment-938055687 then I'd suggest just making every core-resolver that's used from config (like !stack_attr stack_group_name) additionally available from jinja with the same name (like {{stack_attr('stack_group_name')}} or even {{'stack_group_name'|stack_attr}}).

Both of the 2 options described above look worse IMHO compared to the obvious and intuitive {{ stack_group_config.name }} or similar, but at least it does achieve the kind of symmetry and config/template decoupling that most people will expect.

[alex-harvey-z3q]

@mattvonrocketstein You have mentioned above that you are running from a fork. Is that still the case? If so, can you show us the exact code you have added that helps me to understand where you are coming from?