WallarooLabs / wally

Distributed Stream Processing
https://www.wallaroolabs.com
Apache License 2.0
1.48k stars 69 forks source link

add a mechanism for using application configuration data #3138

Open aturley opened 4 years ago

aturley commented 4 years ago

The behavior of a Wallaroo application is controlled by the application code, the incoming data, and the data that is stored as part of a state computation. There are places where this model falls short, and it would be nice to have a way to deal with this.

As a specific example, in our C ONNX application there are several global variables in the C library that store information about the environment and session. These variables are set when a model is sent into the application via a data source. In a way it is "state", but it isn't really "state" in the way that we think of state in a state computation. It isn't changing over time (though there is a desire to be able to dynamically update it), and it should ideally only be set once. It would be useful to have a mechanism in place to allow the application implementor to more cleanly deal with setting and using these kinds of values.