sassoftware / viya4-deployment

This project contains Ansible code that creates a baseline in an existing Kubernetes environment for use with the SAS Viya Platform, generates the manifest for an order, and then can also deploy that order into the Kubernetes environment specified.
Apache License 2.0
70 stars 64 forks source link

fix: (IAC-956) Handle Special Characters for Orchestration Credentials #421

Closed jarpat closed 1 year ago

jarpat commented 1 year ago

Changes

Changes were made to how the task that parses the default credentials from {{ DEPLOY_DIR }}/sas-bases/base/secrets.yaml, we now are properly escaping parsed credentials to handle the case where they may contain Jinja2 special characters. Documentation was also added to tell users to add the !unsafe denotation who have special Jinja2 character sequences in credentials the put in their ansible-vars. This tells ansible not to "templatize" the values.

Tests

Methodology:

The task that parses and incorrectly escapes credentials with special Jinja2 characters is orchestration tooling - set registry credentials if using defaults. This task will only run if a user did not define ORCHESTRATION_CR_USER/ORCHESTRATION_CR_PASSWORD and will look inside the {{ DEPLOY_DIR }}/sas-bases/base/secrets.yaml file for the credentials that were provided by SAS for the default CR.

These credentials that are randomly generated and provided by SAS and there is no method as far as I am aware to customize it, so we will have to "indirectly" test the credentials with the special Jinja2 characters. I will initially deploy my own Harbor container registry to an internal VM instance and use mirrormgr to mirror SAS Viya platform images to it. Then inside Harbor I will create users that have the "special" characters within their password.

My process then will be to pause mid-execution and manually update the credential entry in {{ DEPLOY_DIR }}/sas-bases/base/secrets.yaml with the user I just created in Harbor and see if the characters get properly escaped. I am expecting that the skopeo download or docker pull of the sas-orchestration image will successful, and we should no longer see a templating error that looks like: original message: template error while templating string...

Scenario Provider Order Cadence Deployment Method DEPLOY Credentials (created this user in my personal harbor instance) Notes
1 Azure ** fast:2020 Ansible FALSE user1:{%foo{%12345A
2 Azure ** fast:2020 Ansible FALSE user2:bar{{12345A
3 Azure ** fast:2020 Ansible FALSE user3:{#testuser{#12345A
4 Azure ** fast:2020 Ansible FALSE user4:#mytest##account#12345A
5 Azure ** fast:2020 Ansible FALSE user5:*+?.^{]$& Reg1 regex special characters
6 Azure ** fast:2020 Ansible FALSE user6:A1{%a%}{{b}}{#c#}#d##
7 Azure ** stable:2023.03 R/S Ansible TRUE Unmodified used cr.sas.com to verified all images were still successfully pulled
8 Azure ** fast:2020 Docker FALSE user1:{%foo{%12345A
9 Azure ** fast:2020 Docker FALSE user2:bar{{12345A
10 Azure ** fast:2020 Docker FALSE user3:{#testuser{#12345A
11 Azure ** fast:2020 Docker FALSE user4:#mytest##account#12345A
12 Azure ** fast:2020 Docker FALSE user5:*+?.^{]$& Reg1 regex special characters
13 Azure ** fast:2020 Docker FALSE user6:A1{%a%}{{b}}{#c#}#d##
14 Azure ** stable:2023.03 R/S Docker TRUE Unmodified used cr.sas.com to verified all images were still successfully pulled

Extra Test

Extra 1.

For users who are defining V4_CFG_CR_PASSWORD/V4_CFG_CR_USER directory in their ansible-vars which could container special Jinja2 characters, we document in CONFIG-VARS.md that they should mark the variable as "!unsafe" meaning that Ansible will not attempt to template the string and leave it as is. Marking the value as unsafe is something that needs to be done at variable definition time meaning it has to happen inside the ansible-vars.yaml, at the moment it does not seem to be able to be changed/updated after the fact. Documentation for unsafe keyword

Example:

V4_CFG_CR_PASSWORD: !unsafe "my{%crpassword"

For this test I will define an unsafe V4_CFG_CR_PASSWORD and ensure that the cr_access.json file correctly gets created and that the image pull secret is able to be successfully used in the cluster.

Scenario Provider Order Cadence Deployment Method DEPLOY Credentials (created this user in my personal harbor instance) Notes
Extra 1 OSS ** fast:2020 Docker TRUE user6:A1{%a%}{{b}}{#c#}#d## defined V4_CFG_CR_PASSWORD with unsafe
thpang commented 1 year ago

In general feeling like folks are going to start asking why we are allowing unsafe passwords given the nature of this fix. Have we thought about creating our own python module to escape the value before use instead of this method?

jarpat commented 1 year ago

In general feeling like folks are going to start asking why we are allowing unsafe passwords given the nature of this fix. Have we thought about creating our own python module to escape the value before use instead of this method?

I am not sure if the "!unsafe" keyword is cause for concern, actually it's probably the opposite since using it will make ansible treat the variable as a string regardless of it's value. This prevents it from becoming a template and leading to code injections. So in a way it actually "increases" the security of a variable

I general I don't think writing a python module to escape any strings values that users put in their should be necessary when Ansible already includes and documents the "!unsafe" functionality that handles this for us.

Snippet from doc:

When handling values returned by lookup plugins, Ansible uses a data type called unsafe to block templating. Marking data as unsafe prevents malicious users from abusing Jinja2 templates to execute arbitrary code on target machines. The Ansible implementation ensures that unsafe values are never templated. It is more comprehensive than escaping Jinja2 with {% raw %} ... {% endraw %} tags.

You can use the same unsafe data type in variables you define, to prevent templating errors and information disclosure. You can mark values supplied by vars_prompts as unsafe. You can also use unsafe in playbooks. The most common use cases include passwords that allow special characters like { or %, and JSON arguments that look like templates but should not be templated. For example: