gojek / heimdall

An enhanced HTTP client for Go
http://gojek.tech
Apache License 2.0
2.63k stars 214 forks source link

Fix backoff strategies #84

Closed btamadio closed 4 years ago

btamadio commented 4 years ago

Resolves https://github.com/gojek/heimdall/issues/82 and Resolves https://github.com/gojek/heimdall/issues/83

Tested by checking timestamps in request logs when requesting from a test server that just returns 500.

func main() {
    fmt.Println("testing constant backoff ...")
    s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
        w.WriteHeader(500)
    }))
    defer s.Close()
    cli := httpclient.NewClient(
        httpclient.WithRetryCount(3),
        httpclient.WithHTTPClient(s.Client()),
        httpclient.WithRetrier(
            heimdall.NewRetrier(
                heimdall.NewConstantBackoff(
                    2*time.Second,
                    2*time.Millisecond,
                ),
            ),
        ),
    )
    cli.AddPlugin(plugins.NewRequestLogger(os.Stdout, os.Stderr))
    _, _ = cli.Get(s.URL, nil)

    fmt.Println("testing exponential backoff...")
    cli = httpclient.NewClient(
        httpclient.WithRetryCount(5),
        httpclient.WithHTTPClient(s.Client()),
        httpclient.WithRetrier(
            heimdall.NewRetrier(
                heimdall.NewExponentialBackoff(
                    1*time.Second,
                    8*time.Second,
                    2,
                    2*time.Millisecond,
                ),
            ),
        ),
    )
    cli.AddPlugin(plugins.NewRequestLogger(os.Stdout, os.Stderr))
    _, _ = cli.Get(s.URL, nil)
}

For constant backoff, expect to see 2 seconds between requests.

For exponential backoff, with min=1 second, max=8 second, exp. factor=2, should see backoffs of 1, 2, 4, 8, 8.

Logs confirm expected behavior:

go run main.go
testing constant backoff ...
21/May/2020 11:08:21 GET http://127.0.0.1:51320 500 [0ms]
21/May/2020 11:08:23 GET http://127.0.0.1:51320 500 [1ms]
21/May/2020 11:08:25 GET http://127.0.0.1:51320 500 [1ms]
21/May/2020 11:08:27 GET http://127.0.0.1:51320 500 [1ms]
testing exponential backoff...
21/May/2020 11:08:29 GET http://127.0.0.1:51320 500 [1ms]
21/May/2020 11:08:30 GET http://127.0.0.1:51320 500 [0ms]
21/May/2020 11:08:32 GET http://127.0.0.1:51320 500 [0ms]
21/May/2020 11:08:36 GET http://127.0.0.1:51320 500 [0ms]
21/May/2020 11:08:44 GET http://127.0.0.1:51320 500 [0ms]
21/May/2020 11:08:52 GET http://127.0.0.1:51320 500 [0ms]
coveralls commented 4 years ago

Coverage Status

Coverage decreased (-8.2%) to 91.837% when pulling 551054cccf5973c7bcd3ad17fe902ef743936468 on btamadio:master into 16e67fd885aa9cb5b9d2257936e8e8c301eba52a on gojek:master.

btamadio commented 4 years ago

🎉