search

grafana / sqlds

A package that assists writing SQL-driven datasources
Apache License 2.0
18 stars 12 forks source link

When formatting as a timeseries, convert to many instead of wide #96

Open iwysiu opened 1 year ago

iwysiu commented 1 year ago

Currently, when we need to convert frames into the timeseries format, we covert the long format (what we get from the queries) into a wide format with the data.LongToWide function. The problem with wide format, which we currently use, is that if labels (string fields) have unique values, it creates a new column for every value. This means, that for data with n points, we end up with a n x n sized frame after converting it. Instead, we should convert to the many format, which would create n separate size 1 data frames, taking up significantly less memory.

The main part of this is that we currently use the data.LongToWide function from grafana-plugin-sdk-go and there isn't an equivalent function for LongToMany. From what I understand, the two formats should behave the same, but we should test that.

Info about formats: https://grafana.com/developers/dataplane/timeseries

issue reported in https://github.com/grafana/support-escalations/issues/6904

reagankan commented 1 year ago

Hello team, any updates on if this will be worked on? AMG customers are reaching out for a fix. Thank you

Brandon-Kimberly commented 1 year ago

Bumping this again. Any updates?

taleena commented 1 year ago

Hello, we are currently working through support coverage with AWS. This is not in place yet, but once it is we will be able to take this on.

bhks commented 10 months ago

Hi @taleena ,

Do we have this fix worked on, customers are waiting for this issue to get fixed ?

sidd328 commented 9 months ago

Hi there, checking in on this issue to see if there is an update?

iwysiu commented 9 months ago

Hi! I'm looking into implementing a fix for this, but I want to understand the use case better. Are the labels distinct for every data point? In that case, what is the use case for formatting it as a time series? It wouldn't be able to graph it as actual time series, it would just appear as a bunch of separate points. Would they be able to use it formated as a table?