Open CharlesJhonson opened 9 months ago
(1) You need to enable tmpfs and set db directory onto it to simulate the speed of the disaggregated memory.
(2) Besides, you need to use my forked repo "https://github.com/ruihong123/NovaLSM", or use "db_bench" branch on the original repo. I made some changes to make the code physically disaggregated.
(3) On the memory server, you need to run "nova_server_main" to enable the disaggregated server.
Thank you so much! I previously pulled the original master branch of NovaLSM and extracted the startup parameters of "nova_server_main" based on "nova_lsm_tutorial_exp.sh", but it has not been able to run.
I will try it now following your instructions.
In addition, should I use "nova_lsm_tutorial_exp.sh" or "nova_lsm_tutorial_backup.sh"? They look very similar.
I used your forked repo "https://github.com/ruihong123/NovaLSM", and coredumped when running the memory serve "nova_server_main". The startup parameters are configured as follows:
“/home/caoq/novaDB” is a tmpfs dir. `#!/bin/bash
home_dir="/home/caoq/NovaTest/NovaLSM-master" config_dir="$home_dir/config"
recordcount="100000" value_size="1024"
ltc_migration_policy="immediate" enable_range_index="true" num_migration_threads="1" num_sstable_replicas="1" level="2" l0_start_compaction_mb=$((4*1024)) subrange_no_flush_num_keys="100" enable_detailed_db_stats="false"
major_compaction_type="sc" major_compaction_max_parallism="1" major_compaction_max_tables_in_a_set="15"
enable_flush_multiple_memtables="false" recover_dbs="false" num_recovery_threads="32" zipfian_dist_file_path="/tmp/zipfian" dist="uniform" enable_subrange="false" num_log_replicas="1" log_record_mode="none" scatter_policy="power_of_two" number_of_ltcs="2" enable_lookup_index="true" l0_stop_write_mb=$((10*1024)) num_memtable_partitions="64" num_memtables="256" num_rdma_bg_workers="2"
db_path="/home/caoq/novaDB/db/nova-db-$recordcount-$value_size"
num_storage_workers="8"
cc_stoc_files_path="/home/caoq/novaDB/db/stoc_files"
max_stoc_file_size_mb="4" sstable_size_mb="2" ltc_num_stocs_scatter_data_blocks="1"
port=$((10000+RANDOM%1000)) nova_servers="192.168.10.205:$port" echo "server: $nova_servers"
server_id=0
mem_pool_size_gb="30"
ltc_config_path="" ltc_config_path="$config_dir/nova-tutorial-config"
cc_nconn_workers="1" num_rdma_fg_workers="1" num_compaction_workers="1" block_cache_mb="1" row_cache_mb="4096" memtable_size_mb="4" cc_log_buf_size="1024"
rdma_max_msg_size=$((256*1024)) rdma_port=$((20000+RANDOM%1000)) rdma_max_num_sends="32" rdma_doorbell_batch_size="8" enable_rdma="true" enable_load_data="true" use_local_disk="false"
ltc_num_stocs_scatter_data_blocks="1"
cmd="stdbuf --output=0 --error=0 ./nova_server_main --ltc_migration_policy=$ltc_migration_policy --enable_range_index=$enable_range_index --num_migration_threads=$num_migration_threads --num_sstable_replicas=$num_sstable_replicas --level=$level --l0_start_compaction_mb=$l0_start_compaction_mb --subrange_no_flush_num_keys=$subrange_no_flush_num_keys --enable_detailed_db_stats=$enable_detailed_db_stats --major_compaction_type=$major_compaction_type --major_compaction_max_parallism=$major_compaction_max_parallism --major_compaction_max_tables_in_a_set=$major_compaction_max_tables_in_a_set --enable_flush_multiple_memtables=$enable_flush_multiple_memtables --recover_dbs=$recover_dbs --num_recovery_threads=$num_recovery_threads --sampling_ratio=1 --zipfian_dist_ref_counts=$zipfian_dist_file_path --client_access_pattern=$dist --memtable_type=static_partition --enable_subrange=$enable_subrange --num_log_replicas=$num_log_replicas --log_record_mode=$log_record_mode --scatter_policy=$scatter_policy --number_of_ltcs=$number_of_ltcs --enable_lookup_index=$enable_lookup_index --l0_stop_write_mb=$l0_stop_write_mb --num_memtable_partitions=$num_memtable_partitions --num_memtables=$num_memtables --num_rdma_bg_workers=$num_rdma_bg_workers --db_path=$db_path --num_storage_workers=$num_storage_workers --stoc_files_path=$cc_stoc_files_path --max_stoc_file_size_mb=$max_stoc_file_size_mb --sstable_size_mb=$sstable_size_mb --ltc_num_stocs_scatter_data_blocks=$ltc_num_stocs_scatter_data_blocks --all_servers=$nova_servers --server_id=$server_id --mem_pool_size_gb=$mem_pool_size_gb --use_fixed_value_size=$value_size --ltc_config_path=$ltc_config_path --ltc_num_client_workers=$cc_nconn_workers --num_rdma_fg_workers=$num_rdma_fg_workers --num_compaction_workers=$num_compaction_workers --block_cache_mb=$block_cache_mb --row_cache_mb=$row_cache_mb --memtable_size_mb=$memtable_size_mb --cc_log_buf_size=$cc_log_buf_size --rdma_port=$rdma_port --rdma_max_msg_size=$rdma_max_msg_size --rdma_max_num_sends=$rdma_max_num_sends --rdma_doorbell_batch_size=$rdma_doorbell_batch_size --enable_rdma=$enable_rdma --enable_load_data=$enable_load_data --use_local_disk=$use_local_disk" echo "$cmd" mkdir -p $cc_stoc_files_path && mkdir -p $db_path && $cmd `
run the bash,and coredump info as follow:
hello,I noticed that experiment "We conduct the experiments mostly on a platform consisting of two servers ..." in the paper.
What I want to ask is how to to config or modify the code and script if test NovaLSM with two local machine ? thank you very much.