search

chrislusf / gleam

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
Apache License 2.0
3.41k stars 291 forks source link

Streaming using Gleam #39

Open lzuwei opened 7 years ago

lzuwei commented 7 years ago

Hi I would like to know if Gleam supports stream processing for through 1) raw sockets 2) kafka 3) aws kinesis

If it is possible, is there an example of how to do it. I am exploring golang alternatives of Spark Streaming.

chrislusf commented 7 years ago
  1. already support socket
  2. added in https://github.com/chrislusf/gleam/commit/c0132073d3f8a06a2bf3e080f3b504b148f438c2
  3. not yet. Should be simple to add if you have an account to test it out.

See this introduction on how to add a new source https://github.com/chrislusf/gleam/wiki/Add-New-Source

zixuanweeei commented 5 years ago

Is the stream processing here equivalent to the definition of stateful computation from Apache Flink?

chrislusf commented 5 years ago

stateless