espertechinc / esper

Esper Complex Event Processing, Streaming SQL and Event Series Analysis
GNU General Public License v2.0
842 stars 259 forks source link

Feature to update/add statements dynamically #288

Closed kengjit closed 1 month ago

kengjit commented 1 month ago

From my current understanding, once an EPL module has been compiled and deployed, there is no way to add or update a statement to that specific EPDeployment during runtime without creating a new EPDeployment. For my application, I would like to update and add new EPL statements dynamically without having to do this, is this currently a feature that I am unaware of?

bernhardttom commented 1 month ago

why? make a concrete example

kengjit commented 1 month ago

My use case is to use Esper as a rule engine in which users are able to add/update new rules. However, following the documentation given, I do not see a way to do this during runtime. Meaning once my statements are compiled and deployed, in order to add new statements to my deployment, I will need to compile and deploy again. Please refer to my example code below for a better picture of my question:

EPCompiled epCompiled;
epCompiled = compiler.compile("@name('my-statement') select name, age from PersonEvent", args);
...
...
EPDeployment deployment;
deployment = runtime.getDeploymentService().deploy(epCompiled);
...
...
// NEW STATEMENT I WANT TO ADD INTO DEPLOYMENT
EPCompiled epCompiledNew;
epCompiledNew = compiler.compile("@name('my-statement') select name, age from PersonEvent;@name('my-new-statement') select name, age, salary from PersonEvent", args);

May I know if from the above example, is there any way for me to add a new statement "@name('my-new-statement') select name, age, salary from PersonEvent" into my deployment after already running deployment = runtime.getDeploymentService().deploy(epCompiled);?

Thank you and I appreciate any help provided on this!

bernhardttom commented 1 month ago

What is the benefit to adding a new statement to an existing deployment as compared to just deploying the new statement by itself?

kengjit commented 1 month ago

My thoughts was that if possible, it would be a much cleaner solution since the frequency of statement additions would be quite high. By compiling and deploying a new statement each time I wish to add one, I foresee 2 main issues that I will encounter: 1) I would be left with multiple deployments which could be quite messy. 2) It does not resolve the need to update my statements Example of a statement update:

// Before Update
"@name('my-statement') select name, age from PersonEvent where age > 25"
// After Update
"@name('my-statement') select name, age from PersonEvent where age > 30"

My ideal implementation aims to compile and deploy a single.epl file which contains all my statements, where I can add/update the .epl to make any changes.

Alternatively, it would be useful if there exists functions such as:

// UPDATE STATEMENT
runtime.getDeploymentService().getStatement(deployment.getDeploymentId(), statementName).update(newEplStatement);
// ADD STATEMENT
runtime.getDeploymentService().addStatement(deployment.getDeploymentId(), statementName, eplStatement);
kengjit commented 1 month ago

Hi @bernhardttom, may I check if there are any updates on this?

bernhardttom commented 3 weeks ago

Notice how the EPL is the exact same and how the only change is "age > 25" versus "age > 30". The only change is the filter-for value.

The right way to approach this is by defining a context. This allows you to keep the same statement deployed and just instantiate/terminate instances for different filter values.

Doc links: tip context