search

custodietmssp / Custodiet-Cloud

Custodiet MSSP Cloud Management Platform
0 stars 0 forks source link

Kafka->Kinesis Bridge #3

Open databeast opened 9 years ago

databeast commented 9 years ago

At this point, we're fairly sure we want Kafka as both the MQ buffer and upstream comm system on the CSO CPE host, and as the native MQ solution but for AWS deployments of C4, it seems to make sense to attach things to an AWS KInesis stream. Assuming this is one way, it seems fairly smart to build a Kafka-Kinesis bridge.

Are we language/performance constrained here? Python acceptable or should this one be a java implementation?

Dr-Syn commented 9 years ago

How much data are we anticipating crossing this bridge, and what would need to be done to the data to get it across correctly?

databeast commented 9 years ago

everything generated from a CPE device, minus complete packet contents. essentially comparable to total log EPS.and volume.