search

MentatInnovations / datastream.io

An open-source framework for real-time anomaly detection using Python, ElasticSearch and Kibana
Apache License 2.0
896 stars 164 forks source link

streamz integration #16

Open canagnos opened 6 years ago

erichocean commented 6 years ago

I assume you're referring to this: https://streamz.readthedocs.io/

What I'm wanting to do is have a WebSocket in Python that connects to a server, and then the server continuously sends metrics to dsio, and dsio sends back any anomalies it finds via the WebSocket.

canagnos commented 6 years ago

Yes, that's the project we are referring to. With regards to web sockets would you connect directly, or via Kafka? Do you have a reason to not want a queue in your stack? We're interested!

erichocean commented 6 years ago

Yeah, I would be connecting directly. We don't want/need the overhead of managing Kafka just for message queuing, we already have an orchestrator node that maintains information about the behavior of our system, and that we want to connect to the dsio library to learn the behavior of our system by feeding it data points, and then notify our orchestrator re: outliers so we can take the appropriate action(s).