Error while load testing smartystreets

szunami commented 4 years ago

Dear SmartyStreets folks,

I have really enjoyed using your tools and have been very impressed with your level of documentation.

In doing some load testing of your service, I begin to notice timeouts when I had 100,000 go routines concurrently hitting the batch API. Here's a snippet, for reference

func TestSmartyStreets(t *testing.T) {
    var wg sync.WaitGroup
    for i := 0; i < 100_000; i++ {
        wg.Add(1)
        go doSomething(t, &wg)
    }
    wg.Wait()
}

func doSomething(t *testing.T, wg *sync.WaitGroup) {
    defer wg.Done()

    authId := os.Getenv("SMARTY_AUTH_ID")
    authToken := os.Getenv("SMARTY_AUTH_TOKEN")
    client := wireup.BuildUSStreetAPIClient(
        wireup.SecretKeyCredential(authId, authToken),
        wireup.MaxRetry(8),
    )

    for i := 1; i <= 10; i++ {
        batch := street.NewBatch()
        batch.Append(&street.Lookup{
            Street:        fmt.Sprintf("%d 8th Avenue", rand.Intn(10000)),
            Secondary:     "",
            City:          "Brooklyn",
            State:         "NY",
            ZIPCode:       "11215",
            MaxCandidates: 1,
        })
        err := client.SendBatch(batch)
        require.NoError(t, err, "error talking to smarty streets")
    }
}

This yielded

Get "https://us-street.api.smartystreets.com/street-address?auth-id=ID&auth-token=TOKEN&candidates=1&city=Brooklyn&state=NY&street=6077+8th+Avenue&zipcode=11215": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

I am ok with requests taking longer to complete, but want to make avoid client failures.

I know I could increase the Client Timeout. I also was wondering if you are open to introducing smarter backoff strategies, in particular exponential backoff with jitter (as described here: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/).

Thoughts? I'd be happy to contribute.

mdwhatcott commented 4 years ago

I'm tempted to ask whether your business use case truly requires 100,000 goroutines concurrently hitting the API...but I won't. ;)

Yes, a smarter backoff strategy would be a welcome addition to the SDK. Here's the current implementation:

We have a helper library for faking out randomness, which could prove useful:

https://godoc.org/github.com/smartystreets/random

szunami commented 4 years ago

A reasonable question for sure. Our use case involves CASSing datasets that include hundreds of millions of addresses. We want to be able to parallelize this work as much as possible and we want to make this process as robust as possible.

Ill have a deeper look at the code tomorrow, thanks for the speedy reply!

On Wed, Aug 26, 2020, 6:42 PM Michael Whatcott notifications@github.com wrote:

I'm tempted to ask whether your business use case truly requires 100,000 goroutines concurrently hitting the API...but I won't. ;)

Yes, a smarter backoff strategy would be a welcome addition to the SDK. Here's the current implementation:

Production Code: https://github.com/smartystreets/smartystreets-go-sdk/blob/master/internal/sdk/retry_client.go

Unit Tests: https://github.com/smartystreets/smartystreets-go-sdk/blob/master/internal/sdk/retry_client_test.go

We have a helper library for faking out randomness, which could prove useful:

https://godoc.org/github.com/smartystreets/random

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/smartystreets/smartystreets-go-sdk/issues/17#issuecomment-681160665, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADH626JRYQYG3W3ZZM3DT3SCWFVFANCNFSM4QMIYRDA .

szunami commented 4 years ago

I'm going to close this, thanks for shepherding @mdwhatcott !

smartystreets / smartystreets-go-sdk

Error while load testing smartystreets #17