sfu-dis / corobase

Coroutine-Oriented Main-Memory Database Engine (VLDB 2021)
MIT License
245 stars 30 forks source link

Simple Serial Example Fails with Allocations #6

Open HodBadichi opened 1 day ago

HodBadichi commented 1 day ago

I'm encountering allocation errors when running the simple serial example. Any suggestions on how to fix this would be appreciated

The huge pages configurations:

cat /proc/sys/vm/nr_hugepages

64

cat /proc/sys/vm/nr_hugepages_mempolicy

64

The command and the output : ./run.sh ./ermia_SI ycsb 10 48 20 "-physical_workers_only=1 -index_probe_only=1 -node_memory_gb=75 -null_log_device=1" "-w C -r 10 -s 1000000000 -t sequential"

  • [[ 7 -lt 5 ]]
  • LOGDIR=/dev/shm/hodbadihi/ermia-log
  • mkdir -p /dev/shm/hodbadihi/ermia-log
  • trap 'rm -f /dev/shm/hodbadihi/ermia-log/*' EXIT
  • exe=./ermia_SI
  • shift
  • workload=ycsb
  • shift
  • sf=10
  • shift
  • threads=48
  • shift
  • runtime=20
  • shift
  • bench=ycsb
  • [[ ycsb != \t\p\c\e ]]
  • [[ ycsb != \y\c\s\b ]]
  • '[' -z ']'
  • logbuf_mb=128
  • echo 'logbuf_mb is unset, using 128' logbuf_mb is unset, using 128
  • options='./ermia_SI -physical_workers_only=1 -index_probe_only=1 -node_memory_gb=75 -null_log_device=1 -verbose=1 -benchmark ycsb -threads 48 -scale_factor 10 -seconds 20 -log_data_dir /dev/shm/hodbadihi/ermia-log -log_buffer_mb=128 -log_segment_mb=16384 -parallel_loading'
  • echo ./ermia_SI -physical_workers_only=1 -index_probe_only=1 -node_memory_gb=75 -null_log_device=1 -verbose=1 -benchmark ycsb -threads 48 -scale_factor 10 -seconds 20 -log_data_dir /dev/shm/hodbadihi/ermia-log -log_buffer_mb=128 -log_segment_mb=16384 -parallel_loading ./ermia_SI -physical_workers_only=1 -index_probe_only=1 -node_memory_gb=75 -null_log_device=1 -verbose=1 -benchmark ycsb -threads 48 -scale_factor 10 -seconds 20 -log_data_dir /dev/shm/hodbadihi/ermia-log -log_buffer_mb=128 -log_segment_mb=16384 -parallel_loading
  • '[' ycsb == tpcc ']'
  • '[' ycsb == tpce ']'
  • '[' ycsb == ycsb ']'
  • ./ermia_SI -physical_workers_only=1 -index_probe_only=1 -node_memory_gb=75 -null_log_device=1 -verbose=1 -benchmark ycsb -threads 48 -scale_factor 10 -seconds 20 -log_data_dir /dev/shm/hodbadihi/ermia-log -log_buffer_mb=128 -log_segment_mb=16384 -parallel_loading -benchmark_options '-w C -r 10 -s 1000000000 -t sequential' WARNING: benchmark built in DEBUG mode!!! PID: 4076230 CC: SI phantom-protection: 0 Settings and properties amac-version-chain: 0 arena-size-mb : 4 benchmark : ycsb command-log : 0 command-logbuf : 16MB coro-tx : 0 coro-batch-schedule: 0 coro-batch-size : 5 scan-use-iterator : 0 enable-perf : 0 index-probe-only : 1 log-buffer-mb : 128 log-dir : /dev/shm/hodbadihi/ermia-log log-ship-by-rdma : 0 log_ship_offset_replay : 0 logbuf-partitions : 64 masstree_internal_node_size: 264 masstree_leaf_node_size : 312 node-memory : 75GB num-threads : 48 numa-nodes : 2 numa-mode : compact perf-record-event : persist-policy : sync physical-workers-only: 1 print-cpu-util : 0 read_view_stat_interval : 0ms read_view_stat_file : /dev/shm/ermia_read_view_stat threadpool : 1 tmpfs-dir : /dev/shm tls-alloc : 1 total-threads : 48 var-encode : yes worker-threads : 48 backoff-txns : 0 chkpt-interval : 10 commit-queue : 25000 enable-chkpt : 0 enable-gc : 0 group-commit : 0 group-commit-size : 4KB log-key-for-update: 0 null-log-device : 1 num-backups : 0 parallel-loading: : 1 recovery-warm-up : none retry-txns : 0 scale-factor : 10 truncate-at-bench-start : 0 wait-for-backups : 1 Exception thrown at /specific/disk1/hodbadihi/MLP/corobase/dbcore/sm-alloc.cpp:81 Exception thrown at /specific/disk1/hodbadihi/MLP/corobase/dbcore/sm-alloc.cpp:81 terminating with uncaught exception of type os_error ./run.sh: line 71: 4076230 Aborted (core dumped) $options -benchmark_options "$2"
  • rm -f '/dev/shm/hodbadihi/ermia-log/*
YongjunHe commented 1 day ago

Please follow the instructions in the README to configure hugepage. Your command will create a database with 1 billion records, requiring tens of GB of memory. But your environment only allows up to 64 pages, and each page is only 2MB.