Open Smithx10 opened 3 years ago
ahopp
commented
3 years ago @Smithx10 Thanks for opening this issue. In general, I think we should consider supporting as many community use cases as we can.
Can you share some more about what you'd like to see in Dashboards? Specifically around streams of timestamped values if that's the primary ask. This is a big topic and I think we should start specific and work out from there!
Smithx10
commented
3 years ago Good Day @ahopp,
In the past I briefly read about "Timelion" which was supposed to be able to support some kind of timeseries data but I imagine that was for only Data within Opensearch.
A google mentioned:
The Timelion app is deprecated in 7.0, replaced by dashboard features. In 7.16 and later, the Timelion app is removed from Kibana. To prepare for the removal of Timelion app, you must migrate Timelion app worksheets to a dashboard.
If Opensearch is open to support multiple, as Grafana called "Datasources", that would be the first abstraction I imagine we would need to consider.
I'd imagine the first step would be to allow configuration of different "Opensearch Datasources" to be configured via the UI. (similar to https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/) Depending on the type of Datasource would allow for different types of visualizations. Those for Indexed data for example and those for Timeseries.
Currently, we use a few different visualization panels in Grafana to view Prometheus Time Series data. For a first go round the simplest and most used would be the Time Series Panel ( https://grafana.com/docs/grafana/latest/panels/visualizations/time-series/ )
I think that would be a good starting place.
If we are to go modular with some kind of datasource / plugin architecture for collecting data from multiple sources... I'm not sure if we would what to create some kind of RPC similar to https://github.com/hashicorp/go-plugin. This approach has worked well for allowing the community to expand the Hashicorp tooling to support lots of different applications. Terraform for example has 1323 providers (https://registry.terraform.io/browse/providers) I'd suppose this would only be used server side for fetching / executing the queries from the different providers.
Sorry if I've gone too much into the weeds / implementation.
Thanks Again for your time.
ryn9
commented
2 years ago Believe it or not - before Timelion was merge into Kibana (and no longer a Kibana App) - it used to support some other data sources. You can see some examples in the documentation here: https://github.com/elastic/timelion/blob/master/FUNCTIONS.md.
dtaivpp
commented
1 year ago @ahopp or @anirudha This was completed in part right? Can we backlink to that work here?
ryn9
commented
1 year ago @dtaivpp - it would be great to get a lot more information about the built in Prometheus integration documented and showcased. I believe its under the "observability" umbrella and don't know if there are plans to incorporate any compatibility with the legacy visualizations
ryn9
commented
1 year ago @anirudha I recently saw the following in demo and was wondering if were were going to be seeing PromQL and PPL available soon as visual elements for dashboards
dtaivpp
commented
1 year ago Hey @ryn9 sorry I am just seeing this now. I agree we need to get more demos together for Prometheus integration. I’m working on getting a Kubernetes cluster together to start playing with it more but… well time haha but let me look around and see if I can find anyone working on this at the moment.
dtaivpp
commented
1 year ago @ryn9 I am not certain but I'd imagine what you saw was as a part of one of these two features:
As far as I am aware, it is probably going to be treated more as an arbitrary data store accessible through ppl rather than a first class data integration but I'll let @anirudha comment otherwise if I am off base.
ryn9
commented
1 year ago @dtaivpp @anirudha there is a lot left to be desired for documentation for Prometheus support.
Admittedly there is this: https://opensearch.org/docs/latest/observing-your-data/prometheusmetrics/
What I am most recently trying to understand is if there is any passthrough promql support as was proposed here: https://github.com/opensearch-project/sql/issues/561
I would imagine there would be more information about Prometheus querying here: https://opensearch.org/docs/latest/search-plugins/sql/index/ But, alas, I cannot find anything helpful
With the announcement that Grafana will be no longer a viable option for many users (https://grafana.com/blog/2021/04/20/grafana-loki-tempo-relicensing-to-agplv3/), there is now a demand for a friendly licensed piece of software to step in. Given the values of this project, perhaps Opensearch-Dashboards can fill this need?