VictoriaMetrics / victorialogs-datasource

Grafana datasource for VictoriaLogs
Apache License 2.0
77 stars 2 forks source link

Build a histogram of log counts. #15

Open turbotankist opened 5 months ago

turbotankist commented 5 months ago

It would be very convenient to be able to build a time histogram to navigate through the logs. But a grafana shows: victorialogs-datasource. This datasource does not support full-range histograms. The graph below is based on the logs seen in the response So it shows something, but not a full count of logs. image

steverweber commented 5 months ago

I assume https://docs.victoriametrics.com/victorialogs/roadmap/ will somewhat resolve this if or when some "Stats calculations" are done server side.

grinapo commented 3 months ago

I think #22 would help you too (and latest version of victorialogs).

grinapo commented 2 months ago

Now, let's get back here. I have tried to generate some Time series based on logs: severity:info | stats by (_time:1m) count() res | sort by (_time) It results data in the form:

Time (stamp), Line(string), labels (dict)
2024-06-28 07:46:00, {"res":"5"}, { "res": "5" }
2024-06-28 07:47:00, {"res":"25"}, { "res": "25" }

but grafana would really like stamp+numeric result in a form of: 2024-06-28 07:46:00, 5, { "res": "5" } instead.

There would be nice to have some example (provided it actually works) how to get time series data from VL.

crocchietta commented 2 months ago

I also tried unsuccessfully to use Time series based on logs. It might be interesting to create a new pipe to reformat the results returned to grafana depending on what you are trying to do.

crocchietta commented 2 months ago

There is not too much reaction on this subject. Is this a feature we can hope for in a future release? Or will it never be possible to generate time series from victorialogs datasource?

Loori-R commented 2 months ago

Hi @crocchietta, we plan to add this feature in the future, but the exact timelines are not yet determined.

crocchietta commented 2 months ago

Thank you for taking the time to respond. Very happy with victoriametrics, I chose to continue with victorialogs rather than using Loki. But this feature is missing and seems important to me. Victorialogs has already evolved a lot. So I'll wait. A roadmap specific to victorialogs-datasource would be interesting.

Thank you to the team for the work done.

tiny-pangolin commented 1 month ago

I have a workaround for visualizations you can use the extract fields and convert field type to work around this issue for now, but it doesn't work for explore and it requires extra steps for each panel which is not ideal.

image

crocchietta commented 1 month ago

@tiny-pangolin : thanks. I had already tried with extract fields but I missed the convert field type transformation at the grafana level. Even though this adds steps in building a panel and requires a bit more computing power for the browser, it will allow me to wait for this feature to be added.