search

srotya / sidewinder

Fast and scalable timeseries database
http://sidewinder.srotya.com
Apache License 2.0
25 stars 9 forks source link

Memory optimization #136

Closed ambud closed 5 years ago

ambud commented 5 years ago

Adding simple memory optimization for functions to avoid buffering all datapoints in heap before applying functions. Functions are now evaluated on a series by series basis to reduce memory utilization.

Also fixing series length requirements for multi-series functions. TSQL e.g. ticker.low=>avg,10368000,smean=>multiseries,5184000,smean

codecov-io commented 5 years ago

Codecov Report

Merging #136 into development will increase coverage by 0.42%. The diff coverage is 40.45%.

Impacted file tree graph

@@                Coverage Diff                @@
##             development     #136      +/-   ##
=================================================
+ Coverage          57.02%   57.44%   +0.42%     
- Complexity          1012     1021       +9     
=================================================
  Files                124      124              
  Lines               5992     6016      +24     
  Branches             713      709       -4     
=================================================
+ Hits                3417     3456      +39     
+ Misses              2365     2341      -24     
- Partials             210      219       +9
Impacted Files Coverage Δ Complexity Δ
...com/srotya/sidewinder/core/analytics/Nd4jTest.java 0% <0%> (ø) 0 <0> (ø) :arrow_down:
...va/com/srotya/sidewinder/core/utils/MiscUtils.java 70.96% <0%> (ø) 48 <0> (ø) :arrow_down:
...sidewinder/core/functions/MultiSeriesFunction.java 0% <0%> (ø) 0 <0> (ø) :arrow_down:
...a/sidewinder/core/api/grafana/GrafanaQueryApi.java 0% <0%> (ø) 0 <0> (ø) :arrow_down:
...rotya/sidewinder/core/functions/ChainFunction.java 91.66% <100%> (ø) 5 <0> (ø) :arrow_down:
.../com/srotya/sidewinder/core/storage/TimeField.java 67.67% <100%> (ø) 29 <0> (ø) :arrow_down:
...a/sidewinder/core/storage/FieldReaderIterator.java 74.07% <43.75%> (-12.77%) 16 <3> (+3)
...om/srotya/sidewinder/core/storage/Measurement.java 70.96% <56.66%> (-1.17%) 55 <0> (ø)
...ava/com/srotya/sidewinder/core/storage/Series.java 88.62% <77.14%> (-2.33%) 65 <11> (+7)
...rotya/sidewinder/core/storage/disk/DiskMalloc.java 92.46% <0%> (-0.84%) 35% <0%> (-1%)
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update fb9a08e...710d2f8. Read the comment docs.