search

findify / s3mock

Embedded S3 server for easy mocking
MIT License
386 stars 107 forks source link

Poor performance on Linux compared to Windows #158

Open pixelstuermer opened 4 years ago

pixelstuermer commented 4 years ago

I use the S3Mock for SpringBoot integration tests and noticed a remarkable performance difference when running the build on the local Windows machine compared to the build on a Linux machine (e.g. Jenkins or a local VM):

The Windows build takes about 3 minutes whereas the Linux build takes about 20 minutes. The test Blobstore uses the in-memory backend and Maven as the build tool.

It seems like starting the S3Mock Blobstore is not the problem, but working with the objects. Please see the following logs and have a look at the timestamps and the operations/methods.

Windows

PutObject

2019-10-22 09:04:40.393  INFO 15348 --- [lt-dispatcher-4] io.findify.s3mock.route.PutObject        : put object localbucket/test.txt (unsigned)
2019-10-22 09:04:40.407  INFO 15348 --- [t-dispatcher-15] io.findify.s3mock.route.PutObject        : put object localbucket/outer/test.txt (unsigned)
2019-10-22 09:04:40.420  INFO 15348 --- [t-dispatcher-11] io.findify.s3mock.route.PutObject        : put object localbucket/outer/inner/test.json (unsigned)

DeleteObjects

2019-10-22 09:04:48.377  INFO 15348 --- [lt-dispatcher-2] io.findify.s3mock.route.DeleteObjects    : deleted object localbucket/test.txt
2019-10-22 09:04:48.377  INFO 15348 --- [lt-dispatcher-2] io.findify.s3mock.route.DeleteObjects    : deleted object localbucket/outer/test.txt
2019-10-22 09:04:48.377  INFO 15348 --- [lt-dispatcher-2] io.findify.s3mock.route.DeleteObjects    : deleted object localbucket/outer/inner/test.json

Linux

PutObject

2019-10-22 09:00:11.498  INFO 29088 --- [t-dispatcher-13] io.findify.s3mock.route.PutObject        : put object localbucket/outer/inner/test.json (unsigned)
2019-10-22 09:00:14.577  INFO 29088 --- [t-dispatcher-13] io.findify.s3mock.route.PutObject        : put object localbucket/outer/test.txt (unsigned)
2019-10-22 09:00:17.648  INFO 29088 --- [t-dispatcher-13] io.findify.s3mock.route.PutObject        : put object localbucket/test.txt (unsigned)

DeleteObjects

2019-10-22 09:02:11.614  INFO 30315 --- [t-dispatcher-11] io.findify.s3mock.route.DeleteObjects    : deleted object localbucket/test.txt
2019-10-22 09:02:11.618  INFO 30315 --- [t-dispatcher-11] io.findify.s3mock.route.DeleteObjects    : deleted object localbucket/outer/test.txt
2019-10-22 09:02:11.618  INFO 30315 --- [t-dispatcher-11] io.findify.s3mock.route.DeleteObjects    : deleted object localbucket/outer/inner/test.json

The PutObject somehow only takes some milliseconds on Windows but about 3 seconds on Linux. Wherease DeleteObjects only takes some milliseconds on Linux as well.

Is there any idea how to speed things up on Linux?