The project contains two parts, the Rust-based Falcon and the external C++-based NVM Hash index, Dash.
After installing Rust, run cargo build
to build Falcon.
Init Dash:
git pull --recurse-submodules
source build.sh
or
git clone https://github.com/madsys-dev/dash.git
source build.sh
For more information, please refer to https://github.com/baotonglu/dash.
Init NBTree:
git pull --recurse-submodules
source build.sh
or
git clone https://github.com/madsys-dev/NBTree.git
source build.sh
For more information, please refer to https://github.com/SJTU-DDST/NBTree
If installing Dash or NBTree is difficult, you can use Rust indexes directly to test Falcon.
We use NVM in DAX mode enabled with the following commands.
mkfs-xfs -m reflink=0 -f /dev/pmem0
mount -t xfs /dev/pmem0 /mnt/pmem0 -o dax
Set the path for database file and index file in configure.py
:
db_file_path = "database file path"
pm_index = "persist index file path"
Falcon use taskset
to pin threads to a single NUMA node. Use numactl -H
to check your own hardware. Make sure the CPU and NVM file at the same numa node.
Set the numa information in configure.py
(see examples below):
numa_set = "taskset -c 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96" # Database File and Index File saved in numa node 0 and numa node 0 includes even cores
or
numa_set = "taskset -c 26-51,78-103" # Database File and Index File saved in numa node 1 and numa node 1 includes cores 26~51 and 78~103
configure.py
The size of the test is configured in src/customer_config.rs
. A smaller workload can be used if there is no enough NVM space (see examples below):
// TPCC_WAREHOUSE is no less than THREAD_COUNT.
pub const TPCC_WAREHOUSE:u64 = 48;
pub const YCSB_TOTAL:u64 = 16*1024*1024;
A smaller index can be used if there is no enough NVM space(see examples below):
// in dash/src/dash.cpp(branch tpcc)
static const size_t pool_size = 1024ul * 1024ul * 1024ul * 16ul;
size_t segment_number = 2048 * 32; // for tpcc
Falcon uses conditional compilation to simulate different systems. In Cargo.toml
, fill in the sysname
variable with the relevant parameters:
sysname = ["Falcon"] or sysname = ["Inp"] or ...
After setting, ycsb.sh
and tpcc.sh
can be used to run the corresponding test.
run_test.py
has already configured all the tests. To run the test, simply use the command python3 run_test.py
. If needed, you can modify run_test.py
for customized settings, such as reducing the number of threads:
"thread_count": [[48]] -> "thread_count": [[32]]
After the test, run python3 collect.py
to collect the result, which will export a result.csv
file.
To generate experiment figures, copy the result.csv
to the appropriate location in result.xlsx
to generate the charts.
There are a total of 182 test cases. With full workloads, each test case takes about 3 minutes to complete. Running the entire test suite will take around 9 to 10 hours. If desired, you can modify the Python code to run only a subset of these tests. Here is the number of test cases for each test:
test_ycsb_nvm() # 10 test cases
test_ycsb_dram() # 6 test cases
test_tpcc_nvm() # 60 test cases
test_tpcc_dram() # 36 test cases
test_ycsb_scal() # 35 test cases
test_tpcc_scal() # 35 test cases
If you can’t install Dash, use run_rust_only_test.py
, collect_rust_only.py
and result_rust_only.xlsx
instead.
Recovery evaluation is only for Falcon + Dash, please set feature dash
and Falcon
in Cargo.toml
.
(1) Run execute ycsb.sh
to create the database.
(2) Kill the process when running ycsb.sh
(use Ctrl+C
or kill
).
(3) Run recovery.sh
to recovery the database.
(4) Kill the recovery process and rerun recovery.sh
for recovery again.
For any questions, please contact us at jizc19@mails.tsinghua.edu.cn
.