[FEATURE REQUEST] provide helper to check if all jobs have been run during tests

[krhubert]

EDITED:

I just found an issue about hooks https://github.com/riverqueue/river/issues/122 and https://github.com/riverqueue/river/discussions/167, but I'm not sure what's the status of it.

---

I want to ensure that everything in the project is tested. Reviews are great for that, but failing tests are even more powerful.

Is there a way to ensure each job has been called at least once? A helper can do this in rivertest. Another options are hooks/interceptors/middlewares. By having those, implementing checks is possible. Or maybe there's currently an API that allows this?

I guess the description might be clear enough this is why I have a code example that demonstrates what I'm talking about but for HTTP.

$ go test -v

--- FAIL: TestRest (0.00s)
    rest_test.go:119: rest test report:
        Total: 1, Tested: 0, Missed: 1

        Missed:
          GET /healthcheck

FAIL
exit status 1

package rest

import (
    "bytes"
    "fmt"
    "net/http"
    "net/http/httptest"
    "sync"
    "testing"

    "github.com/labstack/echo/v4"
)

func TestRest(t *testing.T) {
    e := echo.New()
    e.GET("/healthcheck", func(c echo.Context) error { return nil })

    reporter := NewReporter()
    ts := httptest.NewServer(e)
    reporter.Register(e)

    _, err := http.Get(ts.URL)
    if err != nil {
        t.Fatal(err)
    }

    if r := reporter.Report(); !r.Success() {
        t.Log(r.String())
        t.Fail()
    }
}

// Reporter reports which routes have been visited
// during testing. It should be used to check
// if all routes have been tested.
type Reporter struct {
    srv     *echo.Echo
    visited map[string]bool
    mx      sync.Mutex
}

// NewReporter creates a new reporter for the given echo server.
func NewReporter() *Reporter {
    return &Reporter{
        visited: make(map[string]bool),
    }
}

// Register registers echo server and its routes.
func (rep *Reporter) Register(srv *echo.Echo) {
    rep.srv = srv

    for _, route := range srv.Routes() {
        if route.Method == http.MethodGet ||
            route.Method == http.MethodPost ||
            route.Method == http.MethodPut ||
            route.Method == http.MethodPatch ||
            route.Method == http.MethodDelete {
            rep.visited[route.Method+" "+route.Path] = false
        }
    }
}

// Handler returns a http.Handler that can be used by httptest package.
func (rep *Reporter) Handler() http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
        ctx := rep.srv.NewContext(req, w)
        rep.srv.Router().Find(req.Method, req.URL.Path, ctx)

        rep.mx.Lock()
        rep.visited[req.Method+" "+ctx.Path()] = true
        rep.mx.Unlock()
        rep.srv.ServeHTTP(w, req)
    })
}

// Report returns a report of visited routes.
func (rep *Reporter) Report() Report {
    report := Report{}

    for route, visited := range rep.visited {
        report.Total++
        if visited {
            report.Tested++
        } else {
            report.Missed++
            report.MissedRoutes = append(report.MissedRoutes, route)
        }
    }

    return report
}

// Report is a report of visited routes.
type Report struct {
    Total  int
    Tested int
    Missed int

    MissedRoutes []string
}

// Success returns true if all routes have been tested.
func (r Report) Success() bool {
    return r.Missed == 0
}

// String returns a string representation of the report.
func (r Report) String() string {
    var buf bytes.Buffer
    fmt.Fprintf(&buf, "rest test report:\n")
    fmt.Fprintf(&buf, "Total: %d, Tested: %d, Missed: %d\n",
        r.Total, r.Tested, r.Missed)

    if len(r.MissedRoutes) > 0 {
        buf.WriteString("\nMissed:\n")
        for _, route := range r.MissedRoutes {
            buf.WriteString("  " + route + "\n")
        }
    }

    return buf.String()
}

[brandur]

Just on the face of it I feel like a "check every job ran at least once" test helper would be a little too specific of a use case to build into something like rivertest.

What do you think about using the job list API with Client.JobList in a post-test block and iterating through them to make sure everything you wanted is in there?

[krhubert]

@brandur As long as I can create a solution that ensures our code is tested, and at the same time a dev can't opt out of that check, then my problem is solved. So this is perfect, thank you for your suggestion.