tonbo-io / tonbo

A portable embedded database using Arrow.
https://tonbo.io
Apache License 2.0
764 stars 42 forks source link

add wal record buffer #194

Closed crwen closed 1 week ago

crwen commented 2 weeks ago

189

Now data wiil be appended to file every time encoding. This PR add a simple buffer to cache record encoded data when write WAL.

Here are the results of a test on my machine(macbook pro 18) with 10000 DynRecords inserted

without buffer with buffer wal disabled
7.747s 0.714s 0.160s
ethe commented 2 weeks ago

Could it be able to become a general buffered writer in fusio? Always using 4KB or 8KB as a chunk, flush to disk, Record bounding type makes it not general.