Open changtimwu opened 8 years ago
http://prometheus.io/ prometheus uses leveldb and facebook forked it into rocksdb. but why was leveldb being developed at the first place? this answers https://www.quora.com/How-is-RocksDB-compared-with-Redis
quite a lot to study (https://github.com/avelino/awesome-go#database) https://www.reddit.com/r/golang/comments/2ycvwu/catena_a_time_series_storage_engine_written_in_go/
influxdb is more like a service. It's based on BoltDB. please see the quote. We’re moving to BoltDB as the only supported storage engine.
It worthies taking look of BoltDB being used by many projects https://github.com/boltdb/bolt/blob/master/README.md#other-projects-using-bolt
more comparsion http://prometheus.io/docs/introduction/comparison/
http://www.slideshare.net/peihsinsu/plan-your-log-platform-google-cloud-platform bigquery + cloud logging(google hosted fluentd)
http://www.splunk.com/content/splunkcom/zh-hant_cn/resources/operational-intelligence.html good concept of operational intelligence
splunk and ELK are alike in function. splunk is not open source.
http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd
https://www.reddit.com/r/sysadmin/comments/36ja02/help_with_an_alternative_to_splunk/ gotta try ELK sometime
http://newsletter.ascc.sinica.edu.tw/news/read_news.php?nid=2078
而台灣警政署,目前也是採用 Splunk 來做客製化的運用,不同權限的使用者所看到的面板也都不同,各管理階層可客製化其面板及產出統計報告,經由客製化後,對於系統管理員而言,trouble shooting 的時間可大幅縮短,讓系統管理員可以將時間花在解決問題,而不是找出問題。
Splunk 如果要做好客製化,則需要回歸 XML 層面及正規化搜尋語法的建置,以上的條件都需要接受教育訓練以及經驗累積,所以在導入建置時期,顧問的技術支援及人員的教育訓練是很重要的,再者,系統管理者的管理經驗也是非常重要的一環,再好的軟體,若是沒有將實際的系統管理經驗導入,除了耗費極大的建置時間,所呈現的事件資訊也可能是沒意義的資訊,而 Splunk 以單日流量計費的方式,如一股腦地將所有的 Log 沒有經過篩選傳入 Splunk 主機,反而會浪費很多成本,這也是導入時必須注意的事項。
http://mis.bankshung.net/2012/11/splunklog.html splunk supports app and plugin. It has its own community.
quite a lot messaging implementations out there nsq is well-known http://nsq.io/
https://github.com/jaehue/anyq
combine nsq and mqtt with fluentd? https://github.com/uken/fluent-plugin-nsq https://github.com/yuuna/fluent-plugin-mqtt
how about just this? http://zhen.org/blog/surgemq-high-performance-mqtt-server-and-client-libraries-in-go/
pretty much alike what we're doing. http://www.prnewswire.com/news-releases/jolata-launches-out-of-stealth-introduces-network-industrys-first-real-time-big-data-analytics-platform-300072108.html https://angel.co/jolata/followers https://www.sdxcentral.com/articles/news/jolata-brings-big-data-tactics-to-network-monitoring/2015/04/ http://www.phoenixdatacom.com/wp-content/uploads/Jolata-Overview-Phoenix-Datacom.pdf
Nice slide: http://www.atis.org/WSTS/papers/5-4_Jolata_Barry_NetworkAnalyticsandRealTime.pdf
Nice slogan An electron microscope for your network Bring big data to network monitoring
good example of multiple AZ HA http://www.infoq.com/cn/news/2015/03/auth0-webtasks
william talks about devops and CI. http://school.soft-arch.net/blog/70841/ci-effect
http://www.slideshare.net/williamyeh/whoscall-realtime-monitoring