search

OryxProject / oryx

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
http://oryx.io
Apache License 2.0
1.79k stars 405 forks source link

Serving layer speed layer and the ecosystem #211

Closed ghost closed 9 years ago

ghost commented 9 years ago

Hi, Here is what I understood from what I read but from implementation I have a few questions. Please let me know if I can configure my system like the below.

  1. 1 server running serving layer
  2. 2 or 1 servers running speed layer , batch , kafka and spark all together
  3. x nodes of hadoop. Streaming data comes to kafka and speed layer using spark builds the model updates kafka topic For batch I feel data has to come from some etl tool storing to hadoop hdfs and batch builds the model using spark stores into kafka topic. Serving layer reads kafka topic and serves the request via rest api from external clients request be it mobile. Please help if the above servers 1,2 and 3 are feasible options.

Thanks

smarthi commented 9 years ago

Please use the mailing lists for user questions - https://groups.google.com/a/cloudera.org/forum/#!forum/oryx-user

This is only for reporting bugs/issues.