tonbo-io / tonbo

A portable embedded database using Arrow.
https://tonbo.io
Apache License 2.0
764 stars 42 forks source link

fix: clear invalid `version_log` #145

Closed KKould closed 1 month ago

KKould commented 1 month ago

https://github.com/tonbo-io/tonbo/issues/142#issuecomment-2339819930 @crwen I implemented the content in the comment in this pr

github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 125213ms | 37598ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 123211ms | 37411ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 51361ms  | 38145ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52018ms  | 38134ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39070ms  | 38217ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 36765ms  | 38198ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 36077ms  | 36811ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32041ms  | 36822ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6392ms | 9442ms  |
|-------------------------------+--------+---------|
| individual writes             | 8183ms | 10585ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 9336ms | 10724ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 8400ms | 10727ms |
|-------------------------------+--------+---------|
| batch writes                  | 8485ms | 7073ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7655ms | 7050ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 8535ms | 7109ms  |
|-------------------------------+--------+---------|
| removals                      | 3080ms | 3810ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 130163ms | 37189ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 127704ms | 37126ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52628ms  | 37340ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52519ms  | 37312ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39241ms  | 37473ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 38047ms  | 37587ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 35931ms  | 36036ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 33227ms  | 36020ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6315ms | 9446ms  |
|-------------------------------+--------+---------|
| individual writes             | 8302ms | 10617ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 9339ms | 10524ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 8429ms | 10540ms |
|-------------------------------+--------+---------|
| batch writes                  | 8498ms | 7131ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7590ms | 7068ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 8497ms | 7103ms  |
|-------------------------------+--------+---------|
| removals                      | 3099ms | 3806ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 133035ms | 37774ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 132167ms | 37722ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52847ms  | 38364ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52728ms  | 38393ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 38889ms  | 38657ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 37834ms  | 38589ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 35327ms  | 37079ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32408ms  | 36984ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6336ms | 9586ms  |
|-------------------------------+--------+---------|
| individual writes             | 8200ms | 10753ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 9353ms | 10648ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 8417ms | 10721ms |
|-------------------------------+--------+---------|
| batch writes                  | 8476ms | 7078ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7639ms | 7076ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 8549ms | 7128ms  |
|-------------------------------+--------+---------|
| removals                      | 3087ms | 3785ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 120940ms | 37496ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 117918ms | 37446ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50549ms  | 37883ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50545ms  | 37894ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39491ms  | 38324ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 37757ms  | 38373ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 35797ms  | 36758ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 33866ms  | 36782ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6344ms | 9446ms  |
|-------------------------------+--------+---------|
| individual writes             | 8273ms | 10615ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 9376ms | 10688ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 8488ms | 10655ms |
|-------------------------------+--------+---------|
| batch writes                  | 8567ms | 7106ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7643ms | 7071ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 8559ms | 7107ms  |
|-------------------------------+--------+---------|
| removals                      | 3098ms | 3809ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 120512ms | 38172ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 117945ms | 38233ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50879ms  | 37150ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50695ms  | 37001ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39552ms  | 37199ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 37655ms  | 37231ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 36008ms  | 35564ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 33885ms  | 35583ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6539ms | 9624ms  |
|-------------------------------+--------+---------|
| individual writes             | 8389ms | 10719ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 9453ms | 10600ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 8468ms | 10582ms |
|-------------------------------+--------+---------|
| batch writes                  | 8612ms | 7135ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7723ms | 7129ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 8617ms | 7186ms  |
|-------------------------------+--------+---------|
| removals                      | 3159ms | 3816ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 131334ms | 37463ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 130463ms | 37314ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52983ms  | 37567ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53057ms  | 37569ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39001ms  | 38056ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 37996ms  | 38067ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 36280ms  | 36489ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32805ms  | 36458ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6511ms | 9715ms  |
|-------------------------------+--------+---------|
| individual writes             | 8517ms | 10920ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8369ms | 10921ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7991ms | 10932ms |
|-------------------------------+--------+---------|
| batch writes                  | 8165ms | 7345ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 8053ms | 7312ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7877ms | 7341ms  |
|-------------------------------+--------+---------|
| removals                      | 3156ms | 3934ms  |
+-------------------------------+--------+---------+