search

srotya / sidewinder

Fast and scalable timeseries database
http://sidewinder.srotya.com
Apache License 2.0
25 stars 10 forks source link

Performance Fixes #125

Closed ambud closed 6 years ago

ambud commented 6 years ago

Tag key value pair separation has caused database performance impact and slowdown by about 20%, additionally memory utilization of Sidewinder needs to be reduced so Heap is freed to scale and store larger number of series.

ambud commented 6 years ago

Following testing is done using Influx Comparison benchmark on 10K scale var and 1 hours worth of data. Heap configuration is 8GB.

  1. String tsbucket to integer tsbucket string: 953MB, 921MB = 32MB Saving 3663c8acce6d6b6f17544a3c809d6710a71015de

  2. String intern for fields (removed due to significant CPU usage) stringL: 953MB, 882MB (877MB) = 75MB Saving

  3. Remove conf from writer; hashmap clone Total 2.6/2.5 GB to 1.75GB = 870MB Saving f7d1e5bf553f7550908492498a99710e32e1c179

  4. String compression for row key 1.75GB to 1.5GB

  5. Rowkey reference in Writer converted to ByteString 1.5GB to 1.26GB

  6. Removed tsbucket reference from writer object 1.26GB to 1.26GB f7d1e5bf553f7550908492498a99710e32e1c179

  7. Convert writer seriesid reference to linkedbytestring 1.26GB to 1.22GB 33f31f850c3adc0afe4d0b4308fb52226bcb6b84

ambud commented 6 years ago

Released in 0.2.1: https://github.com/srotya/sidewinder/releases/tag/0.2.1