Investigate ways to run long-running Spark streaming apps (not Spark batch actions) from Celos.
Maybe a new kind of external service is enough, maybe we also need a new schedule type (because a streaming app doesn't have slots).
First goal is to be able to monitor a long-running streaming app's status (running/failed, nothing more) in Celos UI.
Also consider deploying Spark apps from Celos CI. Maybe we need a new subdirectory spark or something in the deployment directory, and new entries in the target files or new cmd line options.
Investigate ways to run long-running Spark streaming apps (not Spark batch actions) from Celos.
Maybe a new kind of external service is enough, maybe we also need a new schedule type (because a streaming app doesn't have slots).
First goal is to be able to monitor a long-running streaming app's status (running/failed, nothing more) in Celos UI.
Also consider deploying Spark apps from Celos CI. Maybe we need a new subdirectory
sparkor something in the deployment directory, and new entries in the target files or new cmd line options.