Fix Docs rst tags {% raw %} and {% endraw %}

dalibor-aleksic-atomia commented 5 years ago

I'm struggling with templating.

Based on documentation here, I can set something like this in config.yaml file

project_code: {% raw %}{{ var.my_project_name }}{% endraw %}
region: eu-west-1

and use it like sceptre --var "my_project_name=test-project" generate dev/vpc.yaml, but I get an error

  File "/home/impress/.local/lib/python2.7/site-packages/yaml/constructor.py", line 138, in construct_mapping
    "found unacceptable key (%s)" % exc, key_node.start_mark)
yaml.constructor.ConstructorError: while constructing a mapping
  in "<unicode string>", line 1, column 15:
    project_code: {{ var.my_project_name }}
                  ^
found unacceptable key (unhashable type: 'dict')
  in "<unicode string>", line 1, column 16:
    project_code: {{ var.my_project_name }}

Also this doesn't work in stack yaml files (ex. config/dev/vpc.yaml) . It works only if I use it like

project_code: {{ var.my_project_name }}
region: eu-west-1

Example from documentation (using {% raw %} at beginning) doesn't work either

{% raw %}
profile: {{ var.profile }}
project_code: {{ var.project_code | default("prj") }}
region: {{ command_path.2 }}
template_bucket_name: {{ environment_variable.TEMPLATE_BUCKET_NAME }}
{% endraw %}

I want to achieve project_code: {{ var.project_name }}_something_{ var.another_variable}, but based on documentation and this error I'm not finding a way.

Used example:

Directory structure

.
├── config
│   ├── config.yaml
│   └── dev
│       └── vpc.yaml
└── templates
└── vpc.template.yaml

config/config.yaml content

project_code: {% raw %}{{ var.my_project_name }}{% endraw %}
region: eu-west-1

config/dev/vpc.yaml content
```
template_path: vpc.template.yaml
```

parameters: TagExample: ThisIsTagExample

- `templates/vpc.template.yaml` content

AWSTemplateFormatVersion: "2010-09-09"

Description: >- Demo CloudFormation Template

Parameters: TagExample: Type: String

Resources: VPC: Type: AWS::EC2::VPC Properties: CidrBlock: 10.10.0.0/16 EnableDnsHostnames: true EnableDnsSupport: true Tags:

Key: Name Value: !Sub '${AWS::StackName}_sceptre-poc-vpc'
Key: TagExample Value: !Ref TagExample
```
- Command: ` sceptre --var "my_project_name=my-test-name" --debug generate dev/vpc.yaml`
- Output:
```
$ sceptre --var "my_project_name=my-test-name" --debug generate dev/vpc.yaml [2019-07-09 17:54:32] - Adding yaml constructors for the entry point groups ['sceptre.hooks', 'sceptre.resolvers'] [2019-07-09 17:54:32] - Added constructor for <class 'sceptre.hooks.asg_scaling_processes.ASGScalingProcesses'> with node tag !asg_scheduled_actions [2019-07-09 17:54:32] - Added constructor for <class 'sceptre.hooks.cmd.Cmd'> with node tag !cmd [2019-07-09 17:54:32] - Added constructor for <class 'sceptre.resolvers.stack_output.StackOutput'> with node tag !stack_output [2019-07-09 17:54:32] - Added constructor for <class 'sceptre.resolvers.file_contents.FileContents'> with node tag !file_contents [2019-07-09 17:54:32] - Added constructor for <class 'sceptre.resolvers.environment_variable.EnvironmentVariable'> with node tag !environment_variable [2019-07-09 17:54:32] - Added constructor for <class 'sceptre.resolvers.stack_output.StackOutputExternal'> with node tag !stack_output_external [2019-07-09 17:54:32] - Reading in 'dev/config.yaml' files... Traceback (most recent call last): File "/home/impress/.local/bin/sceptre", line 11, in sys.exit(cli()) File "/home/impress/.local/lib/python2.7/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/home/impress/.local/lib/python2.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/impress/.local/lib/python2.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/impress/.local/lib/python2.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/impress/.local/lib/python2.7/site-packages/click/core.py", line 555, in invoke return callback(args, kwargs) File "/home/impress/.local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func return f(get_current_context(), *args, *kwargs) File "/home/impress/.local/lib/python2.7/site-packages/sceptre/cli/helpers.py", line 36, in decorated return func(args, kwargs) File "/home/impress/.local/lib/python2.7/site-packages/sceptre/cli/template.py", line 64, in generate_command plan = SceptrePlan(context) File "/home/impress/.local/lib/python2.7/site-packages/sceptre/plan/plan.py", line 34, in init all_stacks, command_stacks = config_reader.construct_stacks() File "/home/impress/.local/lib/python2.7/site-packages/sceptre/config/reader.py", line 214, in construct_stacks self.read(path.join(directory, self.context.config_file)) File "/home/impress/.local/lib/python2.7/site-packages/sceptre/config/reader.py", line 269, in read this_config = self._recursive_read(directory_path, filename, config) File "/home/impress/.local/lib/python2.7/site-packages/sceptre/config/reader.py", line 307, in _recursive_read config = self._recursive_read(parent_directory, filename, stack_group_config) File "/home/impress/.local/lib/python2.7/site-packages/sceptre/config/reader.py", line 310, in _recursive_read child_config = self._render(directory_path, filename, stack_group_config) or {} File "/home/impress/.local/lib/python2.7/site-packages/sceptre/config/reader.py", line 353, in _render config = yaml.safe_load(rendered_template) File "/home/impress/.local/lib/python2.7/site-packages/yaml/init.py", line 162, in safe_load return load(stream, SafeLoader) File "/home/impress/.local/lib/python2.7/site-packages/yaml/init.py", line 114, in load return loader.get_single_data() File "/home/impress/.local/lib/python2.7/site-packages/yaml/constructor.py", line 45, in get_single_data return self.construct_document(node) File "/home/impress/.local/lib/python2.7/site-packages/yaml/constructor.py", line 54, in construct_document for dummy in generator: File "/home/impress/.local/lib/python2.7/site-packages/yaml/constructor.py", line 404, in construct_yaml_map value = self.construct_mapping(node) File "/home/impress/.local/lib/python2.7/site-packages/yaml/constructor.py", line 214, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/home/impress/.local/lib/python2.7/site-packages/yaml/constructor.py", line 138, in construct_mapping "found unacceptable key (%s)" % exc, key_node.start_mark) yaml.constructor.ConstructorError: while constructing a mapping in "", line 1, column 15: project_code: {{ var.my_project_name }} ^ found unacceptable key (unhashable type: 'dict') in "", line 1, column 16: project_code: {{ var.my_project_name }} ^

Sceptre version: 2.1.5 Python: 2.7.15+

cornerman commented 5 years ago

Based on documentation here, I can set something like this in config.yaml file

project_code: {% raw %}{{ var.my_project_name }}{% endraw %}

Oh, I think the {% raw %} and {% endraw %} strings might be a formatting error in the documentation. You can safely ignore them and leave them out.

For concatenating two vars, you can do:

project_code: {{ var.project_name }}_something_{{ var.another_variable }}

ngfgrant commented 5 years ago

Hi @dalibor-aleksic-atomia

Can you try without the {% raw %} and {% endraw %} it looks like a rst added those when building the docs

dalibor-aleksic-atomia commented 5 years ago

It works :)

config/config.yaml content

project_code: {{ var.my_project_name }}_something_{{ var.another_variable }}
region: eu-west-1

Output:

$ sceptre --var "my_project_name=my-test-name" --var "another_variable=test2" --debug generate dev/vpc.yaml
[2019-07-09 18:14:51] - Adding yaml constructors for the entry point groups ['sceptre.hooks', 'sceptre.resolvers']
[2019-07-09 18:14:51] - Added constructor for <class 'sceptre.hooks.asg_scaling_processes.ASGScalingProcesses'> with node tag !asg_scheduled_actions
[2019-07-09 18:14:51] - Added constructor for <class 'sceptre.hooks.cmd.Cmd'> with node tag !cmd
[2019-07-09 18:14:51] - Added constructor for <class 'sceptre.resolvers.stack_output.StackOutput'> with node tag !stack_output
[2019-07-09 18:14:51] - Added constructor for <class 'sceptre.resolvers.file_contents.FileContents'> with node tag
!file_contents
[2019-07-09 18:14:51] - Added constructor for <class 'sceptre.resolvers.environment_variable.EnvironmentVariable'>
with node tag !environment_variable
[2019-07-09 18:14:52] - Added constructor for <class 'sceptre.resolvers.stack_output.StackOutputExternal'> with node tag !stack_output_external
[2019-07-09 18:14:52] - Reading in 'dev/config.yaml' files...
[2019-07-09 18:14:52] - Config: {'region': 'eu-west-1', 'project_path': '/tmp/poc-template-error', 'project_code':
'my-test-name_something_test2', 'stack_group_path': 'dev'}
[2019-07-09 18:14:52] - Reading in 'dev/vpc.yaml' files...
[2019-07-09 18:14:52] - Config: {'template_path': 'vpc.template.yaml', 'stack_group_path': 'dev', 'parameters': {'TagExample': 'ThisIsTagExample'}, 'region': 'eu-west-1', 'project_path': '/tmp/poc-template-error', 'project_code':
'my-test-name_something_test2'}
[2019-07-09 18:14:52] - Generate edges for graph
[
{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Parameters": {
        "TagExample": {
            "Type": "String"
        }
    },
    "Description": "Demo CloudFormation Template",
    "Resources": {
        "VPC": {
            "Type": "AWS::EC2::VPC",
            "Properties": {
                "EnableDnsSupport": true,
                "CidrBlock": "10.10.0.0/16",
                "EnableDnsHostnames": true,
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": {
                            "Fn::Sub": "${AWS::StackName}_sceptre-poc-vpc"
                        }
                    },
                    {
                        "Key": "TagExample",
                        "Value": {
                            "Ref": "TagExample"
                        }
                    }
                ]
            }
        }
    }
}
]

ngfgrant commented 5 years ago

I've updated the issue title and will see if anyone can get can this fixed shortly

Sceptre / sceptre

Fix Docs rst tags {% raw %} and {% endraw %} #785