This is probably what @jdenen was intending but to clarify "installation" in this context I think this should be a separate chart (our own or someone else's) added to the Hindsight dependencies as an optional addition. Logging is very org- and environment-specific and choosing an open source option should be easily configurable for a user whose company is heavily invested in Datadog.
I would also recommend this lean toward the more K8s-y EFD stack of Elasticsearch, FluentD/Fluentbit, Kibana as Fluentbit is much lighter weight and more efficient than Logstash and having a Daemonset of Fluentbit log shippers all sending to a single instance of FluentD for aggregating is going to be a better use of cluster resources then using Logstash. Elastic is already enough of a beast.
Ultimately, our responsibility should be to ensure all of Hindsight's service components are sending their logs to the STDOUT channel and then any helm work we do logging should be very decoupled and primarily to make our own deployments easier or bootstrap newcomers with no prior Kubernetes deployments.
jdenen
commented
4 years ago
This is probably what @jdenen was intending but to clarify "installation" in this context I think this should be a separate chart (our own or someone else's) added to the Hindsight dependencies as an optional addition. Logging is very org- and environment-specific and choosing an open source option should be easily configurable for a user whose company is heavily invested in Datadog.
I would also recommend this lean toward the more K8s-y EFD stack of Elasticsearch, FluentD/Fluentbit, Kibana as Fluentbit is much lighter weight and more efficient than Logstash and having a Daemonset of Fluentbit log shippers all sending to a single instance of FluentD for aggregating is going to be a better use of cluster resources then using Logstash. Elastic is already enough of a beast.
Ultimately, our responsibility should be to ensure all of Hindsight's service components are sending their logs to the STDOUT channel and then any helm work we do logging should be very decoupled and primarily to make our own deployments easier or bootstrap newcomers with no prior Kubernetes deployments.