search

aws / aws-dax-go

AWS DAX SDK for the Go programming language. https://aws.amazon.com/dynamodb/dax
Apache License 2.0
47 stars 49 forks source link

Intermittent stall and failure #59

Open Quantumplation opened 3 weeks ago

Quantumplation commented 3 weeks ago

Hello,

When requesting many dynamodb items in parallel (in my case, 50) through the DAX client, the DAX client seemingly deadlocks and ultimately fails with the following error:

UnknownError: unknown error
caused by: file already closed

I did some initial investigation, and it appears this error is thrown here: https://github.com/aws/aws-dax-go/blob/cb4d531a8928bb60f30411eeb9e8eb9d835793b7/dax/internal/client/tubepool.go#L123

My hypothesis is that there is a deadlock, the tubepool gets closed because of some kind of idle check, and then the deadlock times out and fails on this error; but I'm not familiar enough with the code base to investigate further.

Quantumplation commented 3 weeks ago

This behavior appears to disappear somewhere between 8 and 16 requests, but that might just be because it comes rarer and rarer.