Bash-based spread tests

[sed-i]

This new charm is a good opportunity to write integration tests with bash, in spread.

A very simple spread task could look like this:

summary: Feature: k6 exposes self-monitoring metrics during test run

details: |
    Some more details about what we want to accomplish with this test

execute: |
    # GIVEN a load test
    juju deploy k6
    juju deploy loki
    juju relate k6 loki

    # WHEN the load test is started
    juju run k6/leader start some-loki-test
    juju wait-for juju wait-for model $MODEL \
        --query='forEach(units, unit => unit.agent-status == "idle") && forEach(applications, app => app.status != "error")' --timeout=10m

    # THEN metrics are exposed on :8080/metrics
    juju exec --unit k6/0 -- curl localhost:8080/metrics \
        | MATCH '^some_metric_we_know_should_be_there'
    curl $(juju status --format=json | jq -r '.applications.k6.address'):8080/metrics \
        | MATCH '^some_metric_we_know_should_be_there'

The task.yaml above assumes we have a fresh juju model ready to go. This can be accomplished economically using some spread hierarchy, for example:

project prepare                          # charmcraft pack
    backend1 prepare                     # snap install juju, juju bootstrap x, etc
        suite1 prepare
            project prepare-each
                backend1 prepare-each
                    suite1 prepare-each
                        task1 prepare    # juju add-model y (create model and set its name in an envvar)
                        task1 execute    # some bashery, or tox -e integration -- --controller=x --model=y --keep-model tests/integration/test_something.py
                        task1 restore    # juju destroy-model y --force --no-wait
                    suite1 restore-each
                backend1 restore-each
            project restore-each
        suite1 restore
    backend1 restore                     # juju kill-controller x
project restore

References

• create the model in prepare-each using the JUJU_MODEL and then set the model name in the task's environment

[Abuelodelanada]

This line can be improved:

juju wait-for juju wait-for model $MODEL \
        --query='forEach(units, unit => unit.agent-status == "idle") && forEach(applications, app => app.status != "error")' --timeout=10m

Like this:

juju wait-for model $MODEL \
        --query='forEach(units, unit => unit.agent-status == "idle") && forEach(applications, app => app.status != "error")' --timeout=10m

Also in these two lines... is MATCH a spread thing? 🤔 ... if not, I would write them this way:

    juju exec --unit k6/0 -- curl localhost:8080/metrics \
        | grep '^some_metric_we_know_should_be_there'
    curl $(juju status --format=json | jq -r '.applications.k6.address'):8080/metrics \
        | grep '^some_metric_we_know_should_be_there'

[MichaelThamm]

If prepare-each creates the model, then why do we create the model again in:

task1 prepare # juju add-model y (create model and set its name in an envvar)