skizzehq / skizze

A probabilistic data structure service and storage
MIT License
771 stars 64 forks source link

Examine DFS options for storage #86

Closed seiflotfy closed 8 years ago

seiflotfy commented 8 years ago

Since we could be dealing with arbitrary huge data structures (assume 125 MB count-min-log) I am not sure it makes sense to using Raft Consensus on sharing the actual "stored data" (not the state) of a sketch across servers. We need to use something like HDFS or SeaWeedFS for that to work.