tonbo-io / tonbo

A portable embedded database using Arrow.
https://tonbo.io
Apache License 2.0
764 stars 42 forks source link

Refactor: use [fusio](https://github.com/tonbo-io/fusio) to implement the storage layer and support multiple storage #163

Closed KKould closed 1 month ago

github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 136978ms | 37863ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 134460ms | 37787ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53292ms  | 37680ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53218ms  | 37731ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 38840ms  | 38093ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 37563ms  | 37933ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 34760ms  | 36319ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 30681ms  | 36181ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6433ms | 9550ms  |
|-------------------------------+--------+---------|
| individual writes             | 8243ms | 10656ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8227ms | 10640ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7771ms | 10674ms |
|-------------------------------+--------+---------|
| batch writes                  | 8058ms | 7137ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7822ms | 7110ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7706ms | 7171ms  |
|-------------------------------+--------+---------|
| removals                      | 3118ms | 3823ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 138091ms | 37846ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 135251ms | 37909ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53065ms  | 38053ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53213ms  | 37889ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 38564ms  | 38157ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 34929ms  | 38028ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 34649ms  | 36626ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 31311ms  | 36543ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6406ms | 9470ms  |
|-------------------------------+--------+---------|
| individual writes             | 8351ms | 10466ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8168ms | 10689ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7694ms | 10589ms |
|-------------------------------+--------+---------|
| batch writes                  | 8009ms | 7102ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7700ms | 7165ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7707ms | 7130ms  |
|-------------------------------+--------+---------|
| removals                      | 3115ms | 3792ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 129046ms | 38181ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 129076ms | 38144ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 52355ms  | 38120ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53120ms  | 37673ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 40735ms  | 37473ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 40130ms  | 37336ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 37260ms  | 35875ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32191ms  | 35816ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6444ms | 9445ms  |
|-------------------------------+--------+---------|
| individual writes             | 8330ms | 10657ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8201ms | 10652ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7787ms | 10590ms |
|-------------------------------+--------+---------|
| batch writes                  | 8021ms | 7150ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7601ms | 7122ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7579ms | 7130ms  |
|-------------------------------+--------+---------|
| removals                      | 3109ms | 3825ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 133243ms | 37035ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 130852ms | 36974ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53094ms  | 36895ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53073ms  | 36839ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39900ms  | 36908ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 39510ms  | 36935ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 36956ms  | 35581ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32172ms  | 35633ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6466ms | 9430ms  |
|-------------------------------+--------+---------|
| individual writes             | 8207ms | 10425ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8141ms | 10445ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7797ms | 10493ms |
|-------------------------------+--------+---------|
| batch writes                  | 7944ms | 7097ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7592ms | 7108ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7357ms | 7140ms  |
|-------------------------------+--------+---------|
| removals                      | 3070ms | 3806ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 128100ms | 37204ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 122432ms | 37067ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 51474ms  | 37348ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 51415ms  | 37423ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39303ms  | 37564ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 38192ms  | 37573ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 35848ms  | 36089ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 31358ms  | 36044ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6489ms | 9522ms  |
|-------------------------------+--------+---------|
| individual writes             | 8282ms | 10622ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8325ms | 10770ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7848ms | 10727ms |
|-------------------------------+--------+---------|
| batch writes                  | 7947ms | 7217ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7563ms | 7174ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7442ms | 7166ms  |
|-------------------------------+--------+---------|
| removals                      | 3098ms | 3822ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 135857ms | 38071ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 128667ms | 38125ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50476ms  | 37992ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53847ms  | 38042ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 38158ms  | 38129ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 38370ms  | 38195ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 36119ms  | 36690ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 31551ms  | 37045ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6374ms | 9510ms  |
|-------------------------------+--------+---------|
| individual writes             | 8386ms | 10699ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8275ms | 10544ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7864ms | 10469ms |
|-------------------------------+--------+---------|
| batch writes                  | 7971ms | 7155ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7550ms | 7165ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7652ms | 7136ms  |
|-------------------------------+--------+---------|
| removals                      | 3105ms | 3793ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 126302ms | 37549ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 123404ms | 37419ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50652ms  | 37382ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 50475ms  | 37323ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39280ms  | 37655ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 33034ms  | 37742ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 30381ms  | 36195ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32431ms  | 36200ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6440ms | 9659ms  |
|-------------------------------+--------+---------|
| individual writes             | 8339ms | 10492ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8245ms | 10476ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7806ms | 10523ms |
|-------------------------------+--------+---------|
| batch writes                  | 8032ms | 7141ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7598ms | 7140ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7622ms | 7130ms  |
|-------------------------------+--------+---------|
| removals                      | 3123ms | 3806ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 136761ms | 37490ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 133886ms | 37780ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53961ms  | 37422ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 53031ms  | 37231ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39582ms  | 37312ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 39151ms  | 37213ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 36138ms  | 35554ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 32139ms  | 35607ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6454ms | 9538ms  |
|-------------------------------+--------+---------|
| individual writes             | 8333ms | 10662ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8254ms | 10723ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7868ms | 10701ms |
|-------------------------------+--------+---------|
| batch writes                  | 7958ms | 7171ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7613ms | 7272ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7443ms | 7282ms  |
|-------------------------------+--------+---------|
| removals                      | 3102ms | 3828ms  |
+-------------------------------+--------+---------+
github-actions[bot] commented 1 month ago

Read:

+--------------------------------------------+----------+---------+
|                                            | tonbo    | rocksdb |
+=================================================================+
| random range reads                         | 129198ms | 39229ms |
|--------------------------------------------+----------+---------|
| random range reads                         | 116346ms | 39126ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 49997ms  | 40353ms |
|--------------------------------------------+----------+---------|
| random range projection reads              | 49972ms  | 40300ms |
|--------------------------------------------+----------+---------|
| random range projection reads (4 threads)  | 39654ms  | 40491ms |
|--------------------------------------------+----------+---------|
| random range projection reads (8 threads)  | 38564ms  | 40539ms |
|--------------------------------------------+----------+---------|
| random range projection reads (16 threads) | 35719ms  | 38945ms |
|--------------------------------------------+----------+---------|
| random range projection reads (32 threads) | 31509ms  | 39040ms |
+--------------------------------------------+----------+---------+
github-actions[bot] commented 1 month ago

Write:

+-------------------------------+--------+---------+
|                               | tonbo  | rocksdb |
+==================================================+
| bulk load                     | 6464ms | 9537ms  |
|-------------------------------+--------+---------|
| individual writes             | 8315ms | 10911ms |
|-------------------------------+--------+---------|
| individual writes (4 threads) | 8208ms | 10880ms |
|-------------------------------+--------+---------|
| individual writes (8 threads) | 7857ms | 10801ms |
|-------------------------------+--------+---------|
| batch writes                  | 7967ms | 7227ms  |
|-------------------------------+--------+---------|
| batch writes (4 threads)      | 7634ms | 7216ms  |
|-------------------------------+--------+---------|
| batch writes (8 threads)      | 7640ms | 7236ms  |
|-------------------------------+--------+---------|
| removals                      | 3161ms | 3855ms  |
+-------------------------------+--------+---------+