local variable 'period' referenced before assignment

[hokiegeek2]

Getting this error in the context of TS anomaly detection on a metricbeat log stream where the timestamps are all from the same day:

results.append(anomaly_detect_ts(pd.Series(item), **self._generateParams())) File "build/bdist.linux-x86_64/egg/anomaly_detection/anomaly_detect_ts.py", line 207, in anomaly_detect_ts UnboundLocalError: local variable 'period' referenced before assignment

I can see in the code why this is happening I am hitting the else clause with granularity of ms and period is never initialized. Logging shows why:

2018-09-19 19:25:49.858000 (data.index[1]) 2018-09-19 19:25:49.180000 (data.index[0])

So from a pure computer science perspective, it's clear what's happening. timediff = data.index[1] - data.index[0] if timediff.days > 0: num_days_per_line = 7 only_last = 'day' if only_last == 'hr' else only_last period = 7 granularity = 'day' elif timediff.seconds / 60 / 60 >= 1: granularity = 'hr' period = 24 elif timediff.seconds / 60 >= 1: granularity = 'min' period = 1440 elif timediff.seconds > 0: granularity = 'sec'

Aggregate data to minutely if secondly

    data = data.resample('60s', label='right').sum()
else:
    granularity = 'ms'

The questions:

Should I change the sampling so that anomaly_detect_ts gets data points at intervals of 1 min or more, or should I consider anomaly_detect_vec for granularity of sec?

Any guidance is very much appreciated, thanks!

--John

[hokiegeek2]

I think at a minimum there should be logic to throw an error, stating that the ms level of granularity is not supported, in analogy to what pycularity does. In addition, need to set period within sec block. Will make changes and test

[hokiegeek2]

Okay, looking at the code more closely and following some testing with a static file of metricbeats data, and I am seeing that the Pandas resample method is working correctly, converting the sec intervals to min:

timestamp 2018-09-20 12:59:00 486581492 2018-09-20 13:00:00 471500234 2018-09-20 13:01:00 486703848 2018-09-20 13:02:00 485937599 2018-09-20 13:03:00 21696736493 2018-09-20 13:04:00 391 2018-09-20 13:05:00 249000260 2018-09-20 13:06:00 463573558 2018-09-20 13:07:00 491965349 2018-09-20 13:08:00 499765826 2018-09-20 13:09:00 482683757 2018-09-20 13:10:00 494652855 2018-09-20 13:11:00 501479188 2018-09-20 13:12:00 507338630 2018-09-20 13:13:00 491230524 2018-09-20 13:14:00 236295619 2018-09-20 13:15:00 0 2018-09-20 13:16:00 311486949 2018-09-20 13:17:00 489250187 2018-09-20 13:18:00 487550592 2018-09-20 13:19:00 472133907 2018-09-20 13:20:00 506761783 2018-09-20 13:21:00 485043873 2018-09-20 13:22:00 485422510 2018-09-20 13:23:00 147205096

But, since the granularity is sec, the period is never set. I imagine I need to set period=1440. I will try that and see what happens.

[hokiegeek2]

Tested error handling for ms granularity and confirmed sec granularity sets period to 1440. Pushed pull request

[hokiegeek2]

@triciascully this is implemented and tested in the master branch of my fork

[hokiegeek2]

Fixed and merged