search

immux / immux1

https://immux.com
0 stars 0 forks source link

Current benchmark of indexed data set #145

Open poppindouble opened 5 years ago

poppindouble commented 5 years ago

Benchspec:

  1. 50000 row_count, 100 num_jsons_per_command, 10 report_period, false verify_correctness
cargo bench --bench indexed_set -- 50000 100 10 0
    1. 100000 row_count, 100 num_jsons_per_command, 10 report_period, false verify_correctness
cargo bench --bench indexed_set -- 100000 100 10 0

Result:

here is the execution time for the above case:

case 1:

took 38ms to execute 10 create_index operations (10/15 done), average 3.80ms per item took 41ms to execute 10 batch_insert operations (10/500 done), average 4.10ms per item took 52ms to execute 10 batch_insert operations (20/500 done), average 5.20ms per item took 61ms to execute 10 batch_insert operations (30/500 done), average 6.10ms per item took 69ms to execute 10 batch_insert operations (40/500 done), average 6.90ms per item took 77ms to execute 10 batch_insert operations (50/500 done), average 7.70ms per item took 75ms to execute 10 batch_insert operations (60/500 done), average 7.50ms per item took 83ms to execute 10 batch_insert operations (70/500 done), average 8.30ms per item took 84ms to execute 10 batch_insert operations (80/500 done), average 8.40ms per item took 91ms to execute 10 batch_insert operations (90/500 done), average 9.10ms per item took 93ms to execute 10 batch_insert operations (100/500 done), average 9.30ms per item took 89ms to execute 10 batch_insert operations (110/500 done), average 8.90ms per item took 91ms to execute 10 batch_insert operations (120/500 done), average 9.10ms per item took 97ms to execute 10 batch_insert operations (130/500 done), average 9.70ms per item took 98ms to execute 10 batch_insert operations (140/500 done), average 9.80ms per item took 107ms to execute 10 batch_insert operations (150/500 done), average 10.70ms per item took 117ms to execute 10 batch_insert operations (160/500 done), average 11.70ms per item took 112ms to execute 10 batch_insert operations (170/500 done), average 11.20ms per item took 137ms to execute 10 batch_insert operations (180/500 done), average 13.70ms per item took 128ms to execute 10 batch_insert operations (190/500 done), average 12.80ms per item took 135ms to execute 10 batch_insert operations (200/500 done), average 13.50ms per item took 148ms to execute 10 batch_insert operations (210/500 done), average 14.80ms per item took 150ms to execute 10 batch_insert operations (220/500 done), average 15.00ms per item took 152ms to execute 10 batch_insert operations (230/500 done), average 15.20ms per item took 156ms to execute 10 batch_insert operations (240/500 done), average 15.60ms per item took 176ms to execute 10 batch_insert operations (250/500 done), average 17.60ms per item took 169ms to execute 10 batch_insert operations (260/500 done), average 16.90ms per item took 169ms to execute 10 batch_insert operations (270/500 done), average 16.90ms per item took 174ms to execute 10 batch_insert operations (280/500 done), average 17.40ms per item took 186ms to execute 10 batch_insert operations (290/500 done), average 18.60ms per item took 195ms to execute 10 batch_insert operations (300/500 done), average 19.50ms per item took 201ms to execute 10 batch_insert operations (310/500 done), average 20.10ms per item took 199ms to execute 10 batch_insert operations (320/500 done), average 19.90ms per item took 199ms to execute 10 batch_insert operations (330/500 done), average 19.90ms per item took 210ms to execute 10 batch_insert operations (340/500 done), average 21.00ms per item took 244ms to execute 10 batch_insert operations (350/500 done), average 24.40ms per item took 238ms to execute 10 batch_insert operations (360/500 done), average 23.80ms per item took 300ms to execute 10 batch_insert operations (370/500 done), average 30.00ms per item took 217ms to execute 10 batch_insert operations (380/500 done), average 21.70ms per item took 232ms to execute 10 batch_insert operations (390/500 done), average 23.20ms per item took 215ms to execute 10 batch_insert operations (400/500 done), average 21.50ms per item took 113ms to execute 10 batch_insert operations (410/500 done), average 11.30ms per item took 112ms to execute 10 batch_insert operations (420/500 done), average 11.20ms per item took 116ms to execute 10 batch_insert operations (430/500 done), average 11.60ms per item took 116ms to execute 10 batch_insert operations (440/500 done), average 11.60ms per item took 123ms to execute 10 batch_insert operations (450/500 done), average 12.30ms per item took 126ms to execute 10 batch_insert operations (460/500 done), average 12.60ms per item took 129ms to execute 10 batch_insert operations (470/500 done), average 12.90ms per item took 137ms to execute 10 batch_insert operations (480/500 done), average 13.70ms per item took 139ms to execute 10 batch_insert operations (490/500 done), average 13.90ms per item took 134ms to execute 10 batch_insert operations (500/500 done), average 13.40ms per item Data verification is skipped

flamegraph:

cargo bench --bench indexed_set -- 50000 100 10 0

case 2:

took 36ms to execute 10 create_index operations (10/15 done), average 3.60ms per item took 43ms to execute 10 batch_insert operations (10/1000 done), average 4.30ms per item took 54ms to execute 10 batch_insert operations (20/1000 done), average 5.40ms per item took 65ms to execute 10 batch_insert operations (30/1000 done), average 6.50ms per item took 66ms to execute 10 batch_insert operations (40/1000 done), average 6.60ms per item took 77ms to execute 10 batch_insert operations (50/1000 done), average 7.70ms per item took 76ms to execute 10 batch_insert operations (60/1000 done), average 7.60ms per item took 87ms to execute 10 batch_insert operations (70/1000 done), average 8.70ms per item took 88ms to execute 10 batch_insert operations (80/1000 done), average 8.80ms per item took 96ms to execute 10 batch_insert operations (90/1000 done), average 9.60ms per item took 97ms to execute 10 batch_insert operations (100/1000 done), average 9.70ms per item took 90ms to execute 10 batch_insert operations (110/1000 done), average 9.00ms per item took 91ms to execute 10 batch_insert operations (120/1000 done), average 9.10ms per item took 95ms to execute 10 batch_insert operations (130/1000 done), average 9.50ms per item took 97ms to execute 10 batch_insert operations (140/1000 done), average 9.70ms per item took 122ms to execute 10 batch_insert operations (150/1000 done), average 12.20ms per item took 113ms to execute 10 batch_insert operations (160/1000 done), average 11.30ms per item took 116ms to execute 10 batch_insert operations (170/1000 done), average 11.60ms per item took 122ms to execute 10 batch_insert operations (180/1000 done), average 12.20ms per item took 127ms to execute 10 batch_insert operations (190/1000 done), average 12.70ms per item took 137ms to execute 10 batch_insert operations (200/1000 done), average 13.70ms per item took 110ms to execute 10 batch_insert operations (210/1000 done), average 11.00ms per item took 121ms to execute 10 batch_insert operations (220/1000 done), average 12.10ms per item took 139ms to execute 10 batch_insert operations (230/1000 done), average 13.90ms per item took 123ms to execute 10 batch_insert operations (240/1000 done), average 12.30ms per item took 124ms to execute 10 batch_insert operations (250/1000 done), average 12.40ms per item took 131ms to execute 10 batch_insert operations (260/1000 done), average 13.10ms per item took 142ms to execute 10 batch_insert operations (270/1000 done), average 14.20ms per item took 147ms to execute 10 batch_insert operations (280/1000 done), average 14.70ms per item took 153ms to execute 10 batch_insert operations (290/1000 done), average 15.30ms per item took 209ms to execute 10 batch_insert operations (300/1000 done), average 20.90ms per item took 180ms to execute 10 batch_insert operations (310/1000 done), average 18.00ms per item took 180ms to execute 10 batch_insert operations (320/1000 done), average 18.00ms per item took 186ms to execute 10 batch_insert operations (330/1000 done), average 18.60ms per item took 198ms to execute 10 batch_insert operations (340/1000 done), average 19.80ms per item took 242ms to execute 10 batch_insert operations (350/1000 done), average 24.20ms per item took 223ms to execute 10 batch_insert operations (360/1000 done), average 22.30ms per item took 170ms to execute 10 batch_insert operations (370/1000 done), average 17.00ms per item took 175ms to execute 10 batch_insert operations (380/1000 done), average 17.50ms per item took 213ms to execute 10 batch_insert operations (390/1000 done), average 21.30ms per item took 177ms to execute 10 batch_insert operations (400/1000 done), average 17.70ms per item took 195ms to execute 10 batch_insert operations (410/1000 done), average 19.50ms per item took 195ms to execute 10 batch_insert operations (420/1000 done), average 19.50ms per item took 220ms to execute 10 batch_insert operations (430/1000 done), average 22.00ms per item took 193ms to execute 10 batch_insert operations (440/1000 done), average 19.30ms per item took 200ms to execute 10 batch_insert operations (450/1000 done), average 20.00ms per item took 200ms to execute 10 batch_insert operations (460/1000 done), average 20.00ms per item took 226ms to execute 10 batch_insert operations (470/1000 done), average 22.60ms per item took 221ms to execute 10 batch_insert operations (480/1000 done), average 22.10ms per item took 214ms to execute 10 batch_insert operations (490/1000 done), average 21.40ms per item took 271ms to execute 10 batch_insert operations (500/1000 done), average 27.10ms per item took 295ms to execute 10 batch_insert operations (510/1000 done), average 29.50ms per item took 243ms to execute 10 batch_insert operations (520/1000 done), average 24.30ms per item took 238ms to execute 10 batch_insert operations (530/1000 done), average 23.80ms per item took 225ms to execute 10 batch_insert operations (540/1000 done), average 22.50ms per item took 245ms to execute 10 batch_insert operations (550/1000 done), average 24.50ms per item took 240ms to execute 10 batch_insert operations (560/1000 done), average 24.00ms per item took 237ms to execute 10 batch_insert operations (570/1000 done), average 23.70ms per item took 263ms to execute 10 batch_insert operations (580/1000 done), average 26.30ms per item took 251ms to execute 10 batch_insert operations (590/1000 done), average 25.10ms per item took 252ms to execute 10 batch_insert operations (600/1000 done), average 25.20ms per item took 326ms to execute 10 batch_insert operations (610/1000 done), average 32.60ms per item took 344ms to execute 10 batch_insert operations (620/1000 done), average 34.40ms per item took 332ms to execute 10 batch_insert operations (630/1000 done), average 33.20ms per item took 263ms to execute 10 batch_insert operations (640/1000 done), average 26.30ms per item took 284ms to execute 10 batch_insert operations (650/1000 done), average 28.40ms per item took 272ms to execute 10 batch_insert operations (660/1000 done), average 27.20ms per item took 275ms to execute 10 batch_insert operations (670/1000 done), average 27.50ms per item took 293ms to execute 10 batch_insert operations (680/1000 done), average 29.30ms per item took 288ms to execute 10 batch_insert operations (690/1000 done), average 28.80ms per item took 362ms to execute 10 batch_insert operations (700/1000 done), average 36.20ms per item took 386ms to execute 10 batch_insert operations (710/1000 done), average 38.60ms per item took 395ms to execute 10 batch_insert operations (720/1000 done), average 39.50ms per item took 345ms to execute 10 batch_insert operations (730/1000 done), average 34.50ms per item took 329ms to execute 10 batch_insert operations (740/1000 done), average 32.90ms per item took 322ms to execute 10 batch_insert operations (750/1000 done), average 32.20ms per item took 329ms to execute 10 batch_insert operations (760/1000 done), average 32.90ms per item took 350ms to execute 10 batch_insert operations (770/1000 done), average 35.00ms per item took 389ms to execute 10 batch_insert operations (780/1000 done), average 38.90ms per item took 448ms to execute 10 batch_insert operations (790/1000 done), average 44.80ms per item took 416ms to execute 10 batch_insert operations (800/1000 done), average 41.60ms per item took 192ms to execute 10 batch_insert operations (810/1000 done), average 19.20ms per item took 275ms to execute 10 batch_insert operations (820/1000 done), average 27.50ms per item took 178ms to execute 10 batch_insert operations (830/1000 done), average 17.80ms per item took 174ms to execute 10 batch_insert operations (840/1000 done), average 17.40ms per item took 179ms to execute 10 batch_insert operations (850/1000 done), average 17.90ms per item took 187ms to execute 10 batch_insert operations (860/1000 done), average 18.70ms per item took 189ms to execute 10 batch_insert operations (870/1000 done), average 18.90ms per item took 193ms to execute 10 batch_insert operations (880/1000 done), average 19.30ms per item took 211ms to execute 10 batch_insert operations (890/1000 done), average 21.10ms per item took 193ms to execute 10 batch_insert operations (900/1000 done), average 19.30ms per item took 193ms to execute 10 batch_insert operations (910/1000 done), average 19.30ms per item took 208ms to execute 10 batch_insert operations (920/1000 done), average 20.80ms per item took 198ms to execute 10 batch_insert operations (930/1000 done), average 19.80ms per item took 209ms to execute 10 batch_insert operations (940/1000 done), average 20.90ms per item took 235ms to execute 10 batch_insert operations (950/1000 done), average 23.50ms per item took 236ms to execute 10 batch_insert operations (960/1000 done), average 23.60ms per item took 240ms to execute 10 batch_insert operations (970/1000 done), average 24.00ms per item took 251ms to execute 10 batch_insert operations (980/1000 done), average 25.10ms per item took 249ms to execute 10 batch_insert operations (990/1000 done), average 24.90ms per item took 276ms to execute 10 batch_insert operations (1000/1000 done), average 27.60ms per item Data verification is skipped

flamegraph:

cargo bench --bench indexed_set -- 100000 100 10 0

Diagnosis:

row_count goes from 50000 to 100000, two flame graphs structure looked similar, I find out that here are the functions that taking significant time:

  1. get_journal
  2. save_instruction_record
  3. IdList::marshal

1 and 3, I think we can't do too much about it: get_journal is mainly related to Rocks performance, IdList::marshal is already being optimized. 85% time of save_instruction_record is spend on bincode::internal::serialize.

Solution:

Previously we are worrying about index_dataset performance, because index is versioned, but after fixing this I think current performance is acceptable.

I think if we still want to make it faster, next step is to remove bincode, and implement our own method to serialize and deserialize InstructionRecord.