adapt goss verifier.yml to be generalized and more flexible

[Sephtex]

Issue Type

• Feature request

Molecule and Ansible details

ansible 2.4.3.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/yi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

$ pip show molecule
Name: molecule
Version: 2.8.2
Summary: Molecule aids in the development and testing of Ansible roles.
Home-page: https://github.com/metacloud/molecule
Author: UNKNOWN
Author-email: UNKNOWN
License: MIT
Location: /usr/local/lib/python2.7/dist-packages
Requires: anyconfig, marshmallow, click-completion, Jinja2, pexpect, click, PyYAML, tabulate, python-gilt, pbr, ansible-lint, tree-format, cookiecutter, yamllint, sh, flake8, colorama, psutil, testinfra

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• OS package

Desired Behavior

The verifier.yml for the goss tests can be made better/more general:

• Add a task to copy templates (because sometimes you want to have some tests which are dynamic). The templates can also be find in the tests directory but have a j2 extension, that way they are distinguishable.
• Use main.yml to be passed to the goss executor which will read all test*.yml files from the test directory. All test files need to start with test, but when having multiple files, they are nicely merged together and warnings if there are duplicates in the tests will be thrown. The output in ansible will also be nicer and easier to read and everything will be printed line by line.

A new proposal for verifier.yml would be the following:

---
# This is an example playbook to execute goss tests.
# Tests need distributed to the appropriate ansible host/groups
# prior to execution by `goss validate`.
#
# The goss ansible module is installed with molecule.  The ANSIBLE_LIBRARY
# path is updated appropriately on `molecule verify`.

# Details about ansible module:
#  - https://github.com/indusbox/goss-ansible

{% raw -%}
- name: Verify
  hosts: all
  vars:
    goss_version: v0.3.2
    goss_arch: amd64
    goss_dst: /usr/local/bin/goss
    goss_sha256sum: 2f6727375db2ea0f81bee36e2c5be78ab5ab8d5981f632f761b25e4003e190ec
    goss_url: "https://github.com/aelsabbahy/goss/releases/download/{{ goss_version }}/goss-linux-{{ goss_arch }}"
    goss_test_directory: /tmp
    goss_format: documentation
  tasks:
    - name: Copy Goss tests
      copy:
        src: "{{ item }}"
        dest: "{{ goss_test_directory }}/{{ item | basename }}"
      with_fileglob:
        - "{{ playbook_dir }}/../default/tests/test_*.yml"
        - "{{ playbook_dir }}/../default/tests/{{ goss_testsuite | default('') }}/test_*.yml"

    - name: Copy Goss template tests
      template:
        src: "{{ item }}"
        dest: "{{ goss_test_directory }}/{{ item | basename | regex_replace('^(.*).j2$', '\\1') }}"
      with_fileglob:
        - "{{ playbook_dir }}/../default/tests/test_*.yml.j2"
        - "{{ playbook_dir }}/../default/tests/{{ goss_testsuite | default('') }}/test_*.yml.j2"

    # the run the goss testing

    - name: Download and install Goss
      become: true
      get_url:
        url: "{{ goss_url }}"
        dest: "{{ goss_dst }}"
        sha256sum: "{{ goss_sha256sum }}"
        mode: 0755

    - name: Create Goss main test file on remote
      copy:
        content: |
          ---
          gossfile:
            test_*.yml: {}
        dest: "{{ goss_test_directory }}/main.yml"

    - name: Execute Goss tests
      command: "goss -g {{ goss_test_directory }}/main.yml validate --format {{ goss_format }}"
      register: test_results
      ignore_errors: true

    - name: Display details about the goss results
      debug:
        msg: "{{ test_results.stdout_lines }}"

    - name: Fail when tests fail
      fail:
        msg: "Goss failed to validate"
      when: test_results.rc != 0
{% endraw -%}

[MarkusTeufelberger]

There are also newer versions of goss available (e.g. v0.3.4), maybe setting it in molecule.yml could also be useful.

[retr0h]

Feel free to submit a PR and we can discuss further there.

[retr0h]

This issue has had no movement in quite some time. Closing out. Please reopen and submit a PR if you wish to improve the verifier play.