Timeseries expression evaluation in rules

[GoogleCodeExporter]

It would be useful if states were recorded as timeseries, and the rule language 
made it simple to evaluate expressions that used those timeseries.  
Specifically, I want a simple way to express rules like:

- Turn the lights off 15 minutes after the last occupancy sensor was tripped
- Turn the sprinklers on if the soil moisture has been below 40% for 1 hour AND 
the derivative for soil moisture is not positive (i.e. it's not going back up)
- Alert me if the garage door has been open for more than 10 minutes and the 
occupancy sensor there hasn't tripped in the last 5.

Many of these rules could be written by keeping state, but it should be 
possible for stateless rules to just examine the variable's history (its 
timeseries).

Original issue reported on code.google.com by david@fastolfe.net on 16 Oct 2011 at 1:28

[GoogleCodeExporter]

Yes, that's indeed something very useful, which I have been thinking about as 
well already.
To base an implementation on timeseries data, we will have to wait for the API 
that Issue 21 will offer.

Original comment by kai.openhab on 16 Oct 2011 at 8:13

• Changed state: Accepted

[GoogleCodeExporter]

Hi guys,

I'm testing how to do that with drools engine. As you know, drools fusion could 
take care of events, temporal reasoning or sliding windows.

This way, you could write a rule like:

------------------------------------
rule "TestingEvents"

when
    $avgTmp : Number( doubleValue > 50 ) from accumulate(
        StateEvent(itemName=="Temperature", $val : newState) over window:length( 5 ),
        average( ((DecimalType)$val).toBigDecimal().doubleValue() ) )
then
    sendCommand("Alert", $avgTmp.toString());
end
--------------------------------------

Which means, 

1. take the last five measures from temperature sensor (whenever they were 
taken), 2. calculate the average value
3. If the avg goes over 50, then send an alert.

Drools takes care of saving in memory the last values (and discards values 
beyond that sliding window).

If you prefer to work with time, it's possible to work with periods (take 
measures from the last two minutes, or the last 1h30m)

More info here: 
http://docs.jboss.org/drools/release/5.3.0.CR1/drools-fusion-docs/html/ch02.html
#d0e1148

But, there is a couple of things/changes.

1. You must declare StateEvent as an event, in your drl.

declare StateEvent
    @role( event )
end

2. You must start the engine in STREAM mode 

I'm testing this right now, i will update.

But the possibilities are impressive and there is no need to work with 
persistence modules (in fact, they can live toguether).

What do you think?

Original comment by juank...@gmail.com on 14 Dec 2011 at 12:46

[GoogleCodeExporter]

I like that this re-uses existing systems to write the rules, but dislike the 
high overhead of the language needed to express something that simple.  

Any thoughts to doing calculations among multiple variables?  
Interpolation/quantization?  In other words, let's say I have independent 
measurements for temperature and humidity, where each measurement may occur 
irregularly or out of sync with the other.  Both of these tend to change in 
smooth curves and thus can be safely interpolated.  Is there (or can you 
envision) a way for me to take a ratio of these two variables at a certain 
point in time, even though I may not have a discrete measurement for one (or 
both) at that point in time?  I might expect the system to interpolate in 
either direction to that point and then compute the ratio.  Or, if I expect to 
do everything at 5 minute intervals, the system might just implicitly 
interpolate and quantize all of the measurements at those points in time.

I almost think we need a separate expression language just to make this easy, 
but it might be sufficient to try and simplify things using intrinsic functions 
like averaging over moving windows, averaging over fixed periods of time, 
quantizing, a few interpolation functions, applying functions like derivatives, 
etc.

Original comment by david@fastolfe.net on 14 Dec 2011 at 4:17

[GoogleCodeExporter]

> I like that this re-uses existing systems to write the rules, but dislike the 
high overhead of the language needed to express something that simple. 

I agree with that statement. The syntax of Drools is the main reason for why I 
have started an alternative rule engine (see 
http://kaikreuzer.blogspot.com/2011/09/using-xbase-for-home-automation-rules.htm
l and the current 0.9.0 code).
Although I see the power of Drools (and your example with temporal reasoning) 
and I would like to re-use existing systems, I am aiming at something more 
integrated. The historical data should not only be available to Drools, but 
also to the new rule engine as well as to the REST API (e.g. to provide data 
for charts) or the XMPP console. The data must hence be stored centrally in 
openHAB, having it only available in Drools is not enough.

My hope is that in http://code.google.com/p/openhab/issues/detail?id=21 we can 
come up with a flexible API, which could also provide hooks for interpolation 
routines etc. This API could then be directly made available to the rule 
engines as well as to the REST or other interfaces.

Original comment by kai.openhab on 14 Dec 2011 at 8:34

[GoogleCodeExporter]

Well, if are looking for solving the use cases at first message, I think that 
drools will satisfy all of these situations.

It's easy and it's powerful, so you have to learn all the drools features and 
syntax to take the best of the drools engine.

I mean, drools provides accumalate functions (sum, avg, min, max, count), but 
also you could make your own accumulate function.

------------------
Accumulate Functions are all pluggable. That means that if needed, custom, 
domain specific functions can easily be added to the engine and rules can start 
to use them without any restrictions. To implement a new Accumulate Functions 
all one needs to do is to create a Java class that implements the 
org.drools.base.acumulators.AccumulateFunction interface and add a line to the 
configuration file or set a system property to let the engine know about the 
new function
------------------

You could add actions to an accumulate expression

------------------
<result pattern> from accumulate( <source pattern>,
                                  init( <init code> ),
                                  action( <action code> ),
                                  reverse( <reverse code> ),
                                  result( <result expression> ) )
-------------

You could use all conditional elements (eval, exists, forall, collect, 
accumulate....

And also you could write your own DSL syntax or functions to make all easier to 
write and maintain, and to reuse commons actions.

Hence, i know it's hard to work with drools. But, it's the best oss brm out 
there and they are working on improve the ui and the rule authoring.

Finally, i don't know your rule engine, Kai, but i guess that you have to 
provide similar features than drools (you know, a rule engine is a rule engine) 
so what's the main reason to write your own?

If it's the hard syntax, we could work to make drools more friendly (maybe 
using guvnor, or decision tables on spreadsheets, or a DSL, or...) and we could 
benefit from all the new features and evolution of this great product.

Original comment by juank...@gmail.com on 15 Dec 2011 at 9:42

[GoogleCodeExporter]

My main concern is that I do not want to base core functionality of openHAB on 
top of Drools - things should be available in openHAB and be integrated into 
Drools and not the other way round.

I do not want openHAB users having to learn Drools with all its complexity. 

>so what's the main reason to write your own?
I want openHAB as simply as possible to use and the editors and syntax are a 
big part of that. The new rule engine provides a highly integrated editor and 
is very lightweight - in contrast to Drools, it could also be run on embedded 
devices. I will soon try to document it and possibly do a screencast about it.

Original comment by kai.openhab on 15 Dec 2011 at 6:49

[GoogleCodeExporter]

Just FTR: The new rule engine is documented here: 
http://code.google.com/p/openhab/wiki/Rules

Original comment by kai.openhab on 2 Feb 2012 at 8:26

[GoogleCodeExporter]

isn't that covered by the new rule engine meanwhile?

Original comment by teichsta on 26 Jul 2012 at 1:58

[GoogleCodeExporter]

Yes, I would say so.

Original comment by kai.openhab on 26 Jul 2012 at 8:06

• Changed state: Implemented
• Added labels: Version-1.0.0
• Removed labels: Version-later