Document testinfra vars management

[watsonb]

Issue Type

• Feature request

Molecule and Ansible details

ansible --version && molecule --version

ansible 2.7.6
  config file = /home/ben/.ansible.cfg
  configured module search path = [u'/home/ben/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ben/venv_ansible-2.7.6/local/lib/python2.7/site-packages/ansible
  executable location = /home/ben/venv_ansible-2.7.6/bin/ansible
  python version = 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0]
molecule, version 2.19.0

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip

Desired Behavior

Warning: this could be considered two issues/features.

1. When working with the provisioner section of molecule.yml for a scenario and feeding TestInfra tests, I notice that I have to violate DRY and explicitly define the values of role variables (e.g. those defined within defaults/main.yml) within the provisioner.inventory.group_vars/host_vars section of molecule.yml for the values to be picked up within TestInfra tests.

2. Moreover, when defining the values of these vars within molecule.yml, I cannot reference the values of other variables using the "{{ }}" Ansible syntax. The double-curly braces seem to be interpreted as string literals and the value is not being substituted.

Examples:

1. Given defaults/main.yml has the following:

win_mssql_instances:
  - instance_name: MSSQLSERVER
    features: SQLENGINE
    instance_collation: SQL_Latin1_General_CP1_CI_AS
    sysadmins:
      - "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
    sqlusers:
      - "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
    setupadmins:
      - "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
    svcaccount: "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
    svcpassword: "{{ ansible_password }}"
    instance_dir: C:\Program Files\Microsoft SQL Server
    installdb_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
    userldb_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
    userdblog_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
    tempdb_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
    tempdblog_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
    backup_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Backup
    databases:
      - mssql_db

If my molecule.yml does NOT also define this, then my TestInfra test:

def test_installed(host):
    """
    Checks that the package is installed
    :param host: instance under test
    :return: None
    """

    # get variables defined in molecule.yml inventory
    ansible_vars = host.ansible.get_variables()
    instances = ansible_vars["win_mssql_instances"]

    for inst in instances:
        path = inst["instance_dir"]
        ansible_args = "path='" + path + "' state=file"

        assert host.ansible("win_file",
                            ansible_args)["changed"] is False

Fails with:

$ molecule verify

>       instances = ansible_vars["win_mssql_instances"]
E       KeyError: 'win_mssql_instances'

However, if I do something like this (repeat myself) in my molecule.yml:

provisioner:
  name: ansible
  inventory:
    group_vars:
      mssql:
        win_mssql_instances:
          - instance_name: MSSQLSERVER
            features: SQLENGINE
            instance_collation: SQL_Latin1_General_CP1_CI_AS
            sysadmins:
              - "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
            sqlusers:
              - "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
            setupadmins:
              - "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
            svcaccount: "{{ inventory_hostname_short[:15] }}\\{{ ansible_user }}"
            svcpassword: "{{ ansible_password }}"
            instance_dir: C:\Program Files\Microsoft SQL Server
            installdb_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
            userldb_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
            userdblog_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
            tempdb_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
            tempdblog_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Data
            backup_path: C:\Program Files\Microsoft SQL Server\MSSQLSERVER\MSSQL\Backup
            databases:
              - mssql_db

Then it works as expected:

$ molecule verify

Verifier completed successfully.

2. Given that I seem to have to do this (repeat myself) for role verification, then I often run into item/issue 2 where I cannot use the double-curly brace notation to reference Ansible magic vars or other vars within the role construct. This is again in relation to molecule verify and NOT molecule converge.

For example:

Given my molecule.yml contains:

provisioner:
  name: ansible
  inventory:
    group_vars:
      mssql:
        win_mssql_tmpdir: "C:\\TEMP\\Microsoft"
        win_mssql_extract_to: "{{ win_mssql_tmpdir}}\\MSSQL"
        win_mssql_extracts_path: "{{ win_mssql_extract_to }}\\Enterprise_Edition_x64\\setup.exe"

And my TestInfra test looks like:

def test_installer_extracted(host):
    """
    Checks that the installer is extracted
    :param host: instance under test
    :return: None
    """

    # get variables defined in molecule.yml inventory
    ansible_vars = host.ansible.get_variables()
    path = ansible_vars["win_mssql_extracts_path"]

    ansible_args = "path='" + path + "' state=file"

    assert host.ansible("win_file",
                        ansible_args)["changed"] is False

You can see the double-curly braces coming through seemingly as a string literal in the informational output:

INFO:testinfra:RUN Ansible('win_file', u"path='{{ win_mssql_extract_to }}\\Enterprise_Edition_x64\\setup.exe' state=file", {'check': True}): {'_ansible_no_log': False, '_ansible_parsed': True, u'changed': False}
tests/test_default.py::test_installer_extracted[ansible://knemol-ar-win-mssql-benlocal.kiewittest.com] PASSED

What's more concerning, it is not properly reporting changed and the test is passing. If I hard-code an invalid file path (like I think I should):

provisioner:
  name: ansible
  inventory:
    group_vars:
      mssql:
        win_mssql_extracts_path: "C:\\fubar\\setup.exe"

I see an error that is apparently swallowed resulting in a passed test:

INFO:testinfra:RUN Ansible('win_file', u"path='C:\\fubar\\setup.exe' state=file", {'check': True}): {'_ansible_no_log': False,
 '_ansible_parsed': True,
 u'changed': False,
 u'msg': u"Get-AnsibleParam: Parameter 'path' has an invalid path 'C:\x0cubar\\setup.exe' specified."}
tests/test_default.py::test_installer_extracted[ansible://knemol-ar-win-mssql-benlocal.kiewittest.com] PASSED

But if I hard-code the path to handle the backslashes correctly for TestInfra:

provisioner:
  name: ansible
  inventory:
    group_vars:
      mssql:
        win_mssql_extracts_path: "C:\\\\fubar\\\\setup.exe"

I get this:

INFO:testinfra:RUN Ansible('win_file', u"path='C:\\\\fubar\\\\setup.exe' state=file", {'check': True}): {'_ansible_no_log': False,
 '_ansible_parsed': True,
 u'changed': False,
 u'msg': u'path C:\\fubar\\setup.exe will not be created'}
    tests/test_default.py::test_installer_extracted[ansible://knemol-ar-win-mssql-benlocal.kiewittest.com] PASSED

These appear to be TestInfra issues and/or Molecule feeding TestInfra, and they could be specific to my Windows-based roles under test, I'm not sure. I know there is a discussion about Molecule Verification in general (e.g. using a verify.yml Ansible playbook to do it). I kind of prefer the notion of a separate tool verifying Ansible rather than using Ansible to verify itself (this recent Boeing/FAA thing comes to mind).

Anyhow, my primary request is the whole not having to repeat yourself in molecule.yml, but we'll see how this shakes out pending future verification discussions.

[decentral1se]

Thanks for raising and sharing your thoughts @watsonb!

When working with the provisioner section of molecule.yml for a scenario and feeding TestInfra tests, I notice that I have to violate DRY and explicitly define the values of role variables (e.g. those defined within defaults/main.yml) within the provisioner.inventory.group_vars/host_vars section of molecule.yml for the values to be picked up within TestInfra tests.

As you have said, this is a testinfra thing and I think we should fix it there. I raised this at testinfra IIUC your issue: https://github.com/philpep/testinfra/issues/345. I would recommend you weigh in there as well and bring your use case and motivations. There's no clear solution yet ...

Moreover, when defining the values of these vars within molecule.yml, I cannot reference the values of other variables using the "{{ }}" Ansible syntax. The double-curly braces seem to be interpreted as string literals and the value is not being substituted.

That sounds like something we should look into.

If we're in agreement about pushing out 1 to https://github.com/philpep/testinfra/issues/345, then let's rename this issue to describe the incorrect role referencing and focus on that. We might start by adding a https://github.com/ansible/molecule/tree/master/test/scenarios to test that it fails the way it should and then dive into the code ... (if anyone is feeling motivate ...)

[decentral1se]

BTW, we're working on getting the latest testinfra integrated (see #2034) and this utilises a new Ansible runner API. You may want to investigate this for the purposes of this issue ...

[1efty]

I found myself in this situation as well. You can create a pytest fixture to limit repetition.

@pytest.fixture(scope='module')
def AnsibleRoleDefaults(host):
  return host.ansible(
    'include_vars', '../../defaults/main.yml'
  }['ansible_facts']

def test_thing(host, AnsibleRoleDefaults):
  var = AnsibleRoleDefaults['var']
  var2 = host.ansible.get_variables()['var']

Due to the way testifnra creates the host fixture, using include_vars will work like it does within a play. So for some cases you might want to use debug msg={{ lookup('template','../../vars/main.yml') | from_yaml }}" this will get you the variables without bothering the variable space.

---

Would it be out of scope for molecule to provide these?

[decentral1se]

Thanks for that update! Molecule shouldn't be providing these fixtures, we need to push for a solution in testinfra that handles this more easily. That's why I opened up https://github.com/philpep/testinfra/issues/345. However, we could think about documenting this for now, as a stop-gap.

[RebelCodeBase]

Here is a pytest plugin that exposes ansible variables and facts, gopass secrets and the ansible python api as pytest fixtures in testinfra tests for molecule: testaid