reverbrain / eblob

Eblob is an append-only low-level IO library, which saves data in blob files. Created as low-level backend for elliptics
GNU Lesser General Public License v3.0
104 stars 29 forks source link

Move key's headers check outside of `eblob_fill_write_control_from_ram` #165

Open shaitan opened 8 years ago

shaitan commented 8 years ago

This check was added in 9685074ff1773b71d324df439a5738407b830e1d and it is intended to determine broken records and incongruity between index and blob.

This check should be moved outside of eblob_fill_write_control_from_ram because eblob_fill_write_control_from_ram is always called under global lock on b->lock and additional seek under this lock leads to degradation of response time.

Our load tests showed follow degradation:

quantile original v0.23.10 v0.23.10 with reverted 9685074ff1773b71d324df439a5738407b830e1d
99% 82.0 ms 26.0 ms (-56.0)
98% 27.0 ms 16.0 ms (-11.0)
95% 26.0 ms 10.0 ms (-16.0)
90% 26.0 ms 7.0 ms (-19.0)
85% 21.0 ms 5.0 ms (-16.0)
80% 18.0 ms 3.0 ms (-15.0)
75% 15.0 ms 3.0 ms (-12.0)
50% 6.0 ms 2.0 ms (-4.0)

In both tests timeout on the client was set to 25 ms.

I will deal with this issue in the near future (1-2 weeks), but until then I'm ready to discuss the proposals where this check can be moved or how it can be transformed.