zz-jason / leanstore

LeanStore is a larger-than-memory database, optimized for NVMe SSD and multi-core CPU, achieves performance close to in-memory systems without having to keep all data in memory.
MIT License
26 stars 2 forks source link

feat: manage all the global resources in a leanstore object, part 1 #45

Closed zz-jason closed 8 months ago

zz-jason commented 8 months ago

Introduce the new API in include/leanstore/Store.hpp, and try to manage all the global resources (files, threads) in an LeanStore object, so that we can have multiple LeanStore objects in one process, which enables multi-tenant. The direct benefit is that we can run parallel tests where each test has its own LeanStore object.

codecov[bot] commented 8 months ago

Codecov Report

Attention: 321 lines in your changes are missing coverage. Please review.

Comparison is base (f20a677) 35.55% compared to head (a5da2c0) 35.86%.

Files Patch % Lines
source/LeanStore.cpp 35.32% 31 Missing and 77 partials :warning:
source/storage/btree/core/BTreeGeneric.cpp 23.85% 55 Missing and 28 partials :warning:
source/storage/btree/core/BTreeGeneric.hpp 17.94% 18 Missing and 14 partials :warning:
source/storage/buffer-manager/BufferManager.cpp 64.44% 6 Missing and 10 partials :warning:
source/storage/btree/TransactionKV.cpp 30.00% 8 Missing and 6 partials :warning:
source/storage/buffer-manager/TreeRegistry.hpp 0.00% 9 Missing :warning:
...ce/storage/btree/core/BTreePessimisticIterator.hpp 42.85% 3 Missing and 5 partials :warning:
...urce/storage/buffer-manager/GuardedBufferFrame.hpp 75.00% 2 Missing and 6 partials :warning:
source/concurrency-recovery/Worker.cpp 58.82% 1 Missing and 6 partials :warning:
source/storage/btree/BasicKV.cpp 45.45% 0 Missing and 6 partials :warning:
... and 11 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #45 +/- ## ========================================== + Coverage 35.55% 35.86% +0.30% ========================================== Files 86 86 Lines 7653 7716 +63 Branches 3677 3691 +14 ========================================== + Hits 2721 2767 +46 - Misses 3232 3240 +8 - Partials 1700 1709 +9 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

zz-jason commented 8 months ago

remaining static variables that may affect the transaction execution in one LeanStore instance:

➜ find source -name "*.hpp" | xargs grep "static " | grep -v "const" | grep -v "void" | grep -v "inline" | grep -v "{" | grep -v "counters" | grep -v "(" | grep -v thread_local | grep -v Tracing | grep -v Debug | grep -v Jump
2:3:5:7:16:24:32:60:75:114:source/LeanStore.hpp:  static FlagListString sPersistedStringFlags;
3:4:6:8:17:25:33:61:76:115:source/LeanStore.hpp:  static FlagListS64 sPersistedS64Flags;
4:5:7:9:19:27:35:65:82:123:source/concurrency-recovery/Worker.hpp:  static std::atomic<u64> sGlobalMinFlushedGSN;
5:6:8:10:20:28:36:66:83:124:source/concurrency-recovery/Worker.hpp:  static std::atomic<u64> sGlobalMaxFlushedGSN;
6:7:9:12:22:30:38:68:87:128:source/concurrency-recovery/Worker.hpp:  static std::unique_ptr<std::atomic<u64>[]> sActiveTxId;
7:8:10:13:23:31:39:69:88:129:source/concurrency-recovery/Worker.hpp:  static std::shared_mutex sGlobalMutex;
8:9:11:14:24:32:40:70:89:130:source/concurrency-recovery/Worker.hpp:  static std::atomic<TXID> sOldestActiveTx;
9:10:12:15:25:33:41:71:90:131:source/concurrency-recovery/Worker.hpp:  static std::atomic<TXID> sOldestActiveShortTx;
10:11:13:16:26:34:42:72:91:132:source/concurrency-recovery/Worker.hpp:  static std::atomic<TXID> sNetestActiveLongTx;
11:12:14:17:27:35:43:73:92:133:source/concurrency-recovery/Worker.hpp:  static std::atomic<TXID> sWmkOfAllTx;
12:13:15:18:28:36:44:74:93:134:source/concurrency-recovery/Worker.hpp:  static std::atomic<TXID> sWmkOfShortTx;