test fails in github despite success in normal env

[zerwes]

TASK [detect diff between result and expected state for ipsec-testdisable] *****
1941
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["cmp", "-s", "cfg/ipsec-testdisable.xml", "ipsec-testdisable-expect.xml"], "delta": "0:00:00.003616", "end": "2024-01-28 05:32:50.026281", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2024-01-28 05:32:50.022665", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

[zerwes]

TASK [print diff between result and expected state for ipsec-testdisable due to failed cmp ...] ***
2028
ok: [localhost] => {
    "diff.stdout_lines": [
        "--- cfg/ipsec-testdisable.xml\t2024-01-28 09:10:12.674354013 +0000",
        "+++ ipsec-testdisable-expect.xml\t2024-01-28 09:08:11.297481736 +0000",
        "@@ -78,11 +78,11 @@",
        "         <address>10.128.0.0</address>",
        "         <netbits>12</netbits>",
        "       </remoteid>",
        "-      <disabled>1</disabled>",
        "       <encryption-algorithm-option>",
        "         <name>aes256</name>",
        "       </encryption-algorithm-option>",
        "       <hash-algorithm-option>hmac_sha1</hash-algorithm-option>",
        "+      <disabled>1</disabled>",
        "     </phase2>",
        "     <phase2>",
        "       <ikeid>1</ikeid>",
        "@@ -102,11 +102,11 @@",
        "         <address>10.224.0.0</address>",
        "         <netbits>12</netbits>",
        "       </remoteid>",
        "-      <disabled>1</disabled>",
        "       <encryption-algorithm-option>",
        "         <name>aes256</name>",
        "       </encryption-algorithm-option>",
        "       <hash-algorithm-option>hmac_sha1</hash-algorithm-option>",
        "+      <disabled>1</disabled>",
        "     </phase2>",
        "   </ipsec>",
        " </opnsense>"
    ]
}

[zerwes]

branch debug-tests

[zerwes]

env in github: Successfully installed MarkupSafe-2.1.4 PyYAML-6.0.1 ansible-9.1.0 ansible-core-2.16.2 cffi-1.16.0 cryptography-42.0.1 jinja2-3.1.3 lxml-5.1.0 packaging-23.2 pycparser-2.21 resolvelib-1.0.1

[zerwes]

last run without errors 5 days ago Successfully installed MarkupSafe-2.1.4 PyYAML-6.0.1 ansible-9.1.0 ansible-core-2.16.2 cffi-1.16.0 cryptography-42.0.0 jinja2-3.1.3 lxml-5.1.0 packaging-23.2 pycparser-2.21 resolvelib-1.0.1

https://github.com/Rosa-Luxemburgstiftung-Berlin/ansible-opnsense/actions/runs/7621612410/job/20758241779

[zerwes]

while comparing the output of the tasks ... the order of the includes changed!

@@ -12,45 +19,46 @@
 ok: [localhost]

 TASK [ipsec test loop ...] *****************************************************
-included: /home/zerwes/git/ansible-opnsense/test/testsimpletaskgeneric.yml for localhost => (item=ipsec-testdisable)
 included: /home/zerwes/git/ansible-opnsense/test/testsimpletaskgeneric.yml for localhost => (item=ipsec-testenable)
 included: /home/zerwes/git/ansible-opnsense/test/testsimpletaskgeneric.yml for localhost => (item=ipsec-test1)
+included: /home/zerwes/git/ansible-opnsense/test/testsimpletaskgeneric.yml for localhost => (item=ipsec-testdisable)

 TASK [register test in all_tests] **********************************************
 ok: [localhost]

-TASK [copy test file ipsec-testdisable] ****************************************
+TASK [copy test file ipsec-testenable] *****************************************
 changed: [localhost]

-TASK [read test vars for ipsec-testdisable] ************************************
+TASK [read test vars for ipsec-testenable] *************************************
 ok: [localhost]

-TASK [set local cfg path to 'cfg/ipsec-testdisable.xml'] ***********************
+TASK [set local cfg path to 'cfg/ipsec-testenable.xml'] ************************
 ok: [localhost]

-TASK [import ipsec task for ipsec-testdisable] *********************************
+TASK [import ipsec task for ipsec-testenable] **********************************
 included: /home/zerwes/git/ansible-opnsense/test/../tasks/ipsec.yml for localhost
...

[zerwes]

task order in last sane state

TASK [report all tests] ********************************************************
ok: [localhost] => {
    "all_tests": {
        "alias": [
            "alias-testunsorted",
            "alias-testenable",
            "alias-testissue49",
            "alias-test1"
        ],
        "dnsserver": [
            "dnsserver-testupdate",
            "dnsserver-test1"
        ],
        "filter": [
            "filter-test1",
            "filter-testifaceunsorted"
        ],
        "ipsec": [
            "ipsec-testdisable",
            "ipsec-test1",
            "ipsec-testenable"
        ],
        "wireguard": [
            "wireguard-test1"
        ]
    }
}

task after enforcing sorted sequence

TASK [report all tests] ********************************************************
ok: [localhost] => {
    "all_tests": {
        "alias": [
            "alias-test1",
            "alias-testenable",
            "alias-testissue49",
            "alias-testunsorted"
        ],
        "dnsserver": [
            "dnsserver-test1",
            "dnsserver-testupdate"
        ],
        "filter": [
            "filter-test1",
            "filter-testifaceunsorted"
        ],
        "ipsec": [
            "ipsec-test1-02-disable",
            "ipsec-test1-03-enable",
            "ipsec-test1"
        ],
        "wireguard": [
            "wireguard-test1"
        ]
    }
}

[zerwes]

closed by #56