Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.
pipeline objects (aka processors) are intanciated at each micro-batch by spark executors. This leads to too many objects creation and GC purge.
we could instead lazily create a pool of processors into a CoreControllerService that handles a pool of objects needed for record processing.
pipeline objects (aka processors) are intanciated at each micro-batch by spark executors. This leads to too many objects creation and GC purge. we could instead lazily create a pool of processors into a CoreControllerService that handles a pool of objects needed for record processing.
may be with https://commons.apache.org/proper/commons-pool/