search

Zuehlke / SHMACK

Automation and convenience in for setting up a S.H.M.A.C.K. stack with DC/OS on AWS
Apache License 2.0
18 stars 8 forks source link

Testcase: Live data ingestion #5

Closed Slartibartfast27 closed 8 years ago

Slartibartfast27 commented 9 years ago

A testcase exists, which ingests current (live twitter data). Assertion: 1000 twitter messages are ingested and can be retrieved from a database (likely cassandra)

mspringma commented 8 years ago

https://github.com/Zuehlke/hackzurich-sensordataanalysis/tree/master/KafkaToCassandra shows an example of Stream processing in Spark writing to Cassandra. Not a minimal test case, but still probably sufficient.

Reading live Twitter stream and reading to Cassandra is implemented in https://github.com/SamuelBucheliZ/SHMACKCamp2016/tree/master/AkkaTwitter/src/main/scala/com/zuehlke/camp/shmack