Scripts is a mechanism to provide more power to the user of Sensetif. It is also a way to provide more functionality in an agile manner, rather than doing a lot of work on the base system. It can almost be viewed as a plugin system, and perhaps in the future it will be expanded into that. All of the functionality of the scripts are happening in the so called Pipes subsystem, written in Java and the GraalVM is used to execute the scripts within the JVM process (although I think GraalVM is native code). Scripts are stored in the Cassandra database, and uses the KeyValues store, to avoid additional database modeling.
This issue only details the work needed for the Grafana side of the equation.
Scripts can be configured to be auto-invoked by;
a. Datapoint value received,
b. Time interval
c. "Condition" occurring
Other types of script triggering may come later.
Time interval is basically hooking the script on to listeners on the sensetif/main/schedule topic with a filter for interval.
Datapoints have a "condition" field in the Processing type, which can have a comma-separated list of scripts in them. Those scripts will be executed on each value being received and the value each script returns will be sent to the sensetif/main/conditions topic (undefined byte[] format).
Finally, scripts can be connected to the sensetif/main/conditions and expected to understand the format that they will receive. The system will only filter by organization and (optionally) by MessageKey (normally has project/subsystem/datapoint in them).
Scripts can either be System-wide, Organization-wide or Project-wide. The user can only update Organization and Project scripts, but will be able to use System scripts, which only users of organization 1 can create and update.
GraalVm is used as the Scripting engine, and the javax.scripting API is not used, due to limitations in needed security.
Script Naming and Referencing is explained in #73
Scripts can only access a small number of Java resources;
All the Cassandra APIs, e.g. Timeseries.class, Datapoints.class, Projects.class, ++ and associated types
Messenger.send() to send messages to a subset of queues
Messenger.notify() for error/info messages.
The MessageKey of the message triggering the script.
For point a. above, the TsDatapoint value that just arrived.
For point b. above, the Schedule (string) that was triggered.
For point c. above, the byte[] of the condition.
Scripts supported will be;
Javascript (recommended)
Python
Ruby
GraalVM needs the JVM to have "-XX:+EnableJVMCI" for native compilation, otherwise the scripts are too slow (100's of milliseconds just to get going).
Other things we need to solve and do;
Grafana UI components
69
68
70
Script Testing/Debug Page, possibly integrated with #69
Grafana Datasource
Fetch System Scripts, Organization Scripts and Project scripts as different APIs
See #73 for script naming in Cassandra and for Pulsar commands.
Store System Scripts, Organization Scripts and Project scripts as different APIs
Scripts is a mechanism to provide more power to the user of Sensetif. It is also a way to provide more functionality in an agile manner, rather than doing a lot of work on the base system. It can almost be viewed as a plugin system, and perhaps in the future it will be expanded into that. All of the functionality of the scripts are happening in the so called Pipes subsystem, written in Java and the GraalVM is used to execute the scripts within the JVM process (although I think GraalVM is native code). Scripts are stored in the Cassandra database, and uses the KeyValues store, to avoid additional database modeling.
This issue only details the work needed for the Grafana side of the equation.
Scripts can be configured to be auto-invoked by; a. Datapoint value received, b. Time interval c. "Condition" occurring
Other types of script triggering may come later.
Time interval is basically hooking the script on to listeners on the sensetif/main/schedule topic with a filter for interval.
Datapoints have a "condition" field in the Processing type, which can have a comma-separated list of scripts in them. Those scripts will be executed on each value being received and the value each script returns will be sent to the sensetif/main/conditions topic (undefined byte[] format).
Finally, scripts can be connected to the sensetif/main/conditions and expected to understand the format that they will receive. The system will only filter by organization and (optionally) by MessageKey (normally has project/subsystem/datapoint in them).
Scripts can either be System-wide, Organization-wide or Project-wide. The user can only update Organization and Project scripts, but will be able to use System scripts, which only users of organization 1 can create and update.
GraalVm is used as the Scripting engine, and the javax.scripting API is not used, due to limitations in needed security.
Scripts can only access a small number of Java resources;
Scripts supported will be;
GraalVM needs the JVM to have "-XX:+EnableJVMCI" for native compilation, otherwise the scripts are too slow (100's of milliseconds just to get going).
Other things we need to solve and do;
Grafana UI components
69
68
70
Grafana Datasource