grafana / opensearch-datasource

Apache License 2.0
27 stars 21 forks source link

Add support for logs volumes #483

Closed psauvage0 closed 3 days ago

psauvage0 commented 3 weeks ago

What this PR does / why we need it:

This add the LogsVolume feature (See https://grafana.com/developers/plugin-tools/tutorials/build-a-logs-data-source-plugin#show-full-range-logs-volume).

This allow the user to see all logs and not just those displayed in the Logs table view.

Here is a preview of the visualization in the Explore View : Explore View

Which issue(s) this PR fixes:

Fixes #352

Special notes for your reviewer:

Code is mostly adapted from ElasticSearch Plugin + Grafana Logs Datasource plugin documentation. Logs present in the screenshot are linux events from https://github.com/logpai/loghub Tests are also adapted from ElasticSearch Plugin, let us know if more tests are required

CLAassistant commented 3 weeks ago

CLA assistant check
All committers have signed the CLA.

jtheory commented 4 days ago

They have 2 other possible PRs to add, but waiting to get feedback on this one -- happy also to help with regular maintenance on the plugin! but need confirmation

taleena commented 4 days ago

Hello @psauvage0 and thank you for the contribution! When looking at the documentation, I see that this API is still under development and a warning that this is not supported for external data sources. Opensearch is an external data source (not within the grafana/grafana repo), so I'm checking with our teams internally to see what other information and context is available.

psauvage0 commented 4 days ago

Hi, thanks for your time @taleena :slightly_smiling_face:

This feature is possible thanks to the work that has been done in this PR from last january : https://github.com/grafana/grafana/pull/80281. You might want to check with the people involved in it. From what I saw in the PR, there should be no problem for external datasource anymore since the goal was precisely to decouple the internal datasources. I'll stay available if you need more info on this.

wilfriedroset commented 4 days ago

As @jtheory said, we are planning to add support for Live tailing (like Loki) via https://github.com/grafana/opensearch-datasource/issues/299 and we want to add a highlight feature so that searched terms are highlighted

matyax commented 3 days ago

This feature is possible thanks to the work that has been done in this PR from last january : https://github.com/grafana/grafana/pull/80281. You might want to check with the people involved in it.

Correct 👍

taleena commented 3 days ago

Thanks for the info here @psauvage0, and to @matyax for taking a look and confirming it's not an issue any longer, and now updating the docs!

The team owning this plugin can now review, we'll see if we can get to it next week. :)

psauvage0 commented 3 days ago

Hi @kevinwcyu and thanks for the review :slightly_smiling_face: I just pushed the commit for the fix