devops related - Githubissues

changtimwu commented 8 years ago

http://www.slideshare.net/williamyeh/whoscall-realtime-monitoring

changtimwu commented 8 years ago

http://prometheus.io/ prometheus uses leveldb and facebook forked it into rocksdb. but why was leveldb being developed at the first place? this answers https://www.quora.com/How-is-RocksDB-compared-with-Redis

quite a lot to study (https://github.com/avelino/awesome-go#database) https://www.reddit.com/r/golang/comments/2ycvwu/catena_a_time_series_storage_engine_written_in_go/

influxdb is more like a service. It's based on BoltDB. please see the quote. We’re moving to BoltDB as the only supported storage engine.

It worthies taking look of BoltDB being used by many projects https://github.com/boltdb/bolt/blob/master/README.md#other-projects-using-bolt

more comparsion http://prometheus.io/docs/introduction/comparison/

changtimwu commented 8 years ago

http://www.slideshare.net/tw_dsconf/elasticsearch-kibana

changtimwu commented 8 years ago

https://github.com/fluent/fluent-bit https://www.youtube.com/watch?v=NYhoFyDwX00

changtimwu commented 8 years ago

http://www.slideshare.net/peihsinsu/plan-your-log-platform-google-cloud-platform bigquery + cloud logging(google hosted fluentd)

changtimwu commented 8 years ago

http://www.splunk.com/content/splunkcom/zh-hant_cn/resources/operational-intelligence.html good concept of operational intelligence

splunk and ELK are alike in function. splunk is not open source.
http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd

changtimwu commented 8 years ago

https://www.reddit.com/r/sysadmin/comments/36ja02/help_with_an_alternative_to_splunk/ gotta try ELK sometime

changtimwu commented 8 years ago

http://newsletter.ascc.sinica.edu.tw/news/read_news.php?nid=2078

而台灣警政署，目前也是採用 Splunk 來做客製化的運用，不同權限的使用者所看到的面板也都不同，各管理階層可客製化其面板及產出統計報告，經由客製化後，對於系統管理員而言，trouble shooting 的時間可大幅縮短，讓系統管理員可以將時間花在解決問題，而不是找出問題。

Splunk 如果要做好客製化，則需要回歸 XML 層面及正規化搜尋語法的建置，以上的條件都需要接受教育訓練以及經驗累積，所以在導入建置時期，顧問的技術支援及人員的教育訓練是很重要的，再者，系統管理者的管理經驗也是非常重要的一環，再好的軟體，若是沒有將實際的系統管理經驗導入，除了耗費極大的建置時間，所呈現的事件資訊也可能是沒意義的資訊，而 Splunk 以單日流量計費的方式，如一股腦地將所有的 Log 沒有經過篩選傳入 Splunk 主機，反而會浪費很多成本，這也是導入時必須注意的事項。

http://mis.bankshung.net/2012/11/splunklog.html splunk supports app and plugin. It has its own community.

changtimwu commented 8 years ago

http://47ron.in/blog/2015/10/23/haproxy-in-the-era-of-microservices.html

changtimwu commented 8 years ago

quite a lot messaging implementations out there nsq is well-known http://nsq.io/

https://github.com/jaehue/anyq

combine nsq and mqtt with fluentd? https://github.com/uken/fluent-plugin-nsq https://github.com/yuuna/fluent-plugin-mqtt

how about just this? http://zhen.org/blog/surgemq-high-performance-mqtt-server-and-client-libraries-in-go/

changtimwu commented 8 years ago

https://github.com/mashape/kong

changtimwu commented 8 years ago

pretty much alike what we're doing. http://www.prnewswire.com/news-releases/jolata-launches-out-of-stealth-introduces-network-industrys-first-real-time-big-data-analytics-platform-300072108.html https://angel.co/jolata/followers https://www.sdxcentral.com/articles/news/jolata-brings-big-data-tactics-to-network-monitoring/2015/04/ http://www.phoenixdatacom.com/wp-content/uploads/Jolata-Overview-Phoenix-Datacom.pdf