verify_response_with not working on mqtt_response

[octathorp]

Hello.

When calling an external function using "verify_response_with" simply nothing happens (with Tavern 2.0.7), but this was working perfectly with Tavern 1.25.2:

stages:
  - name: Send config
    mqtt_publish:
      topic: "{topic_config:s}"
      json: !include json/Config.json
    mqtt_response:
      topic: "{topic_config_ack:s}"
      json:
        debug: !anything
        status: 1
      timeout: 5
      verify_response_with:
        function: testing_utils:verifySetup

def verifySetup(response):
    with open("log.txt", "w") as log:
        log.write("test")

File "log.txt" is created when using Tavern 1.25.2 but nothing happens when using 2.0.7 nor 2.0.6.

At the same time, even if I try to refer to a non-existing function, the test simply passes, but with version 1.25.2 this was being checked in runtime:

AttributeError: module 'testing_utils' has no attribute 'verifyNOTSetup'. Did you mean: 'verifySetup'?

Is that a bug or something changed in most recent versions?

Thank you.

[octathorp]

Seems to be related with "check_ext_functions" call on response.py;

check_ext_functions(response_block.get("verify_response_with", None))

As multiple mqtt_response checks were implemented response_block turned into a json array containing each one of the response checks:

{
    "mqtt_responses": [{
        "topic": "mytopic/config/ack",
        "json": {
            "debug": "debug info",
            "status": 1
        },
        "timeout": 5,
        "verify_response_with": {
            "function": "testing_utils:verifySetup"
        }
    }]
}

So simple calling

response_block.get("verify_response_with", None)

will always return "None" as its necessary to first get "mqtt_responses" to get the response checks array, then get the element from the array, then get "verify_response_with";

check_ext_functions(response_block.get("mqtt_responses")[0].get("verify_response_with", None))

For sure, not getting first element in a hardcoded way, but iterating over the array to get every "verify_response_with" block:

mqtt_responses = response_block.get("mqtt_responses")
for mqtt_response in mqtt_responses:
    check_ext_functions(mqtt_response.get("verify_response_with", None))

[michaelboulton]

Release in 2.6.0