matrix-org / complement

Matrix compliance test suite
Apache License 2.0
61 stars 52 forks source link

Add should package #663

Closed kegsay closed 1 year ago

kegsay commented 1 year ago

The should package is a more lenient form of the must package, in that it does not fail the test if the assertion fails. Instead, these functions return an error, which the test can then handle.

This is particularly useful in cases where a test needs to retry a certain request. Previously, you had to hand-roll the checks on the response because using must.MatchResponse would fail the test, even though it might have been okay as the next retry may pass the matchers.

Fixes https://github.com/matrix-org/complement/issues/546

Tests need to be revisited to see if this can be used in more places.

An example of this. Before:

res = alice.DoFunc(
    t,
    "GET",
    []string{"_matrix", "client", "v3", "rooms", roomID, "aliases"},
    client.WithRetryUntil(
        1*time.Second,
        func(res *http.Response) bool {
            if res.StatusCode != 200 {
                return false
            }
            eventResBody := client.ParseJSON(t, res)
            matcher := match.JSONKeyEqual("aliases", []interface{}{roomAlias})
            err := matcher(eventResBody)
            if err != nil {
                t.Log(err)
                return false
            }
            return true
        },
    ),
)

After:

res = alice.Do(
    t,
    "GET",
    []string{"_matrix", "client", "v3", "rooms", roomID, "aliases"},
    client.WithRetryUntil(
        1*time.Second,
        func(res *http.Response) bool {
            _, err := should.MatchResponse(res, match.HTTPResponse{
                StatusCode: 200,
                JSON: []match.JSON{
                    match.JSONKeyEqual("aliases", []interface{}{roomAlias}),
                },
            })
            if err != nil {
                t.Log(err)
                return false
            }
            return true
        },
    ),
)