compressPeriodicGaps function

[deniszh]

This PR contains function named compressPeriodicGaps() which can help with removing periodic gaps (i.e. None values) from series, (recalculating start, stop and step), which could be useful for users with incorrect resolution setup for some metrics. Especially, that can be useful for derivative functions (like derivative(), nonNegativeDerivative() and perSecond()) which are not working properly with series with gaps. Please check test test_compressPeriodicGaps for illustration how compressPeriodicGaps() works. You can use summarize(..., '<desired interval>', 'last') function to have same effect (thanks to @DanCech for that suggestion) but compressPeriodicGaps() will try to guess proper step by first three non-null values, and it just do nothing with series if guessing failed.

Also, this patch contains support for quite experimantal variable AUTOCOMPRESS_GAPS_IN_DERIVATIVE_FUNCTIONS which if set to True will automatically apply compressGaps() to all argument of derivative functions above. It's a hack, but it's really useful for customers who want to have working derivative functions over series with gaps, without changing many hundreds of dashboards.

I also removing pyhash from Python 3.x tests temporarily because of issue in pyhash which I couldn't fix for now (pipping setuptools<58 for some reason not working). I'll put it back when pyhash release fixed version.

[codecov-commenter]

Codecov Report

Merging #2720 (90908fb) into master (ff764e8) will decrease coverage by 1.87%. The diff coverage is 78.57%.

@@            Coverage Diff             @@
##           master    #2720      +/-   ##
==========================================
- Coverage   79.00%   77.12%   -1.88%     
==========================================
  Files          88      176      +88     
  Lines        9442    18968    +9526     
  Branches     2005     4032    +2027     
==========================================
+ Hits         7460    14630    +7170     
- Misses       1688     3810    +2122     
- Partials      294      528     +234     

Impacted Files	Coverage Δ
webapp/graphite/render/functions.py	95.05% <78.04%> (-0.33%)	:arrow_down:
webapp/graphite/settings.py	75.48% <100.00%> (+0.11%)	:arrow_up:
webapp/graphite/umsgpack.py	0.00% <0.00%> (-42.01%)	:arrow_down:
webapp/graphite/util.py	80.40% <0.00%> (-0.82%)	:arrow_down:
graphite/account/migrations/0001_initial.py	100.00% <0.00%> (ø)
graphite/browser/urls.py	100.00% <0.00%> (ø)
graphite/dashboard/models.py	90.00% <0.00%> (ø)
graphite/render/grammar.py	100.00% <0.00%> (ø)
graphite/readers/multi.py	92.30% <0.00%> (ø)
graphite/readers/whisper.py	89.39% <0.00%> (ø)
... and 84 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ff764e8...90908fb. Read the comment docs.

[deniszh]

Thanks, found a bug because of suggections. Fixed.

[deniszh]

Thanks! Merging

[deniszh]

💔 Some backports could not be created

Status	Branch	Result
❌	1.1.x	Cherrypick failed because the selected commit was a merge commit. Please try again by specifying the parent with the mainline argument:

backport --mainline

or:

backport --mainline

Or refer to the git documentation for more information: https://git-scm.com/docs/git-cherry-pick#Documentation/git-cherry-pick.txt---mainlineparent-number|

To backport manually run: node scripts/backport --pr 2720. For more info read the Backport documentation