search

johannesboyne / gofakes3

A simple fake AWS S3 object storage (used for local test-runs against AWS S3 APIs)
MIT License
361 stars 84 forks source link

Switch to active boltdb fork #54

Closed igungor closed 3 years ago

igungor commented 3 years ago

s5cmd test suite uses gofakes3 extensively while interacting with S3 API. Thank you very much for this project; it's been very useful.

We're using boltdb as the S3 backend. One of our tests uses S3 CopyObject API, but when we run it along with others tests in parallel, we see random crashes. I don't have a minimal repro unfortunately but I can reproduce the crash like below:

git clone git@github.com:peak/s5cmd.git
cd s5cmd && git checkout gofakes3-bolt-crash
go test -parallel=2 -count=1 -v -run='(TestCopySingleS3ObjectToLocal$|TestCopyMultipleS3ObjectsToS3_Issue70$)' ./e2e

Crash log is here: https://gist.github.com/igungor/70239af4df7b75132e84ac1b48a550c7 Also I created a new branch in s5cmd to demonstrate the problem. You could checkout the logs here: https://github.com/peak/s5cmd/runs/2341425465

Then I switched boltdb that gofakes3 use to the active fork mainted by the etcd team to test if the problem is still there. Fortunately, it's been solved. There are no random crashes.

What do you think about changing boltdb to https://github.com/etcd-io/bbolt? If it sounds reasonable to you, I can create a PR for it.

johannesboyne commented 3 years ago

Hi @igungor - just a couple of weeks ago I had this already in mind but did not want to change a running system if not required. But this seems to be a great moment to switch to the active fork! It would be awesome and much appreciated if you could create a PR!