Amazing panel - A bit Slow though

[karnamonkster]

Hi,

Firstly i would like to thank you for creating such an amazing panel for Grafana. This is what i was looking for a really long time. I did installed it with my installation and using it with the MySQL database. The browsing within the Edit Panel seem to be slow. Is there anything that we need to do before using the same to reduce the latency. Another thing, Does it work with ES as datasource?

Thanks once again & Great Job !!

[michaeldmoore]

I'm delighted you found it useful - and managed to setup the data source etc and get it working. Not sure about the slowness you talk about - I guess this might be a function of the amount of data you are processing, otherwise everything seems to respond instantly for me.

Maybe take a look at the query in the MySQL query client first and make sure it isn't taking long there.

The current version only supports 'table' type data sets (as opposed to 'time-series' - which I am working on right now), so somehow you have to make the mySQL query select a single value for each named metric - this might be a challenge depending on your ultimate data source. My idea in for this control imagined a set of metrics (such as stock tickers etc.) with values (prices?) changing at more or less random times. The query would then return the latest value of each of these metrics 'as of' some defined moment of time. Most likely, the current time, hence this means the latest value - still, I can imagine moving the query window back through history and hence setting the 'as-of' date/time as something other than the current latest values. Next up would be some sort of aggregation support, so the values might be some sort of average of the values existing during the query time window - this might get quite complicated though so it's all just an idea for now.

Anyway, generating the latest value for a set of randomly updating data points can be quite a complex task for MySQL, maybe this is your problem. If so, the time-series support I'm working on might come to the rescue, as the aggregation will then be done in the client. The downside will be that the result set returned by mySQL will have ALL the values, not just the latest values - so the network traffic will necessarily by much larger. Probably not an issue if the mySQL server is running on the same PC as Grafana.

On Sun, Jan 28, 2018 at 8:07 PM, karnamonkster notifications@github.com wrote:

Hi,

Firstly i would like to thank you for creating such an amazing panel for Grafana. This is what i was looking for a really long time. I did installed it with my installation and using it with the MySQL database. The browsing within the Edit Panel seem to be slow. Is there anything that we need to do before using the same to reduce the latency.

Thanks once again & Great Job !!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/michaeldmoore/michaeldmoore-multistat-panel/issues/2, or mute the thread https://github.com/notifications/unsubscribe-auth/ADjVrpA4UrlFC5N-SmhTXUwbHhY96b68ks5tPUP5gaJpZM4RwClE .

-- "That's not grey - I've just come from San Francisco with some flour in my hair"

[michaeldmoore]

Heads up - I'm just getting round to releasing a major update - and one that will hopefully satisfy the Grafana developers to add it to the standard repository etc.

Included in this is support for table queries that return more than one record per item - and thinking about it, I suspect this is something that you might be having trouble with.

You see, the basic idea of multistat is that it uses a table query to return a set of data, with at minimum, a label and a value (plus an optional datetimestamp and any other metadata). Each label creates a bar, the length and color etc. of which is set by the value.

One problem occurs when there are multiple records for a given label - the current version tries to plot every one as a separate bar, overlaying on other bars with the same label and creating at best, a mess. The version I'm about to release handles this case, automatically selecting just the last record for any given label. This cleans up the display and doesn't waste time trying to plot multiple bars etc. Ideally, the query itself would eliminate multiple rows per label, though writing such a query can be challenging, depending on the data source.

In my experience, the plugin is lightning fast. I suspect your query returned a huge number of rows for each label - this would explain why it appeared slow.

If you are interested, wait for a few days while Grafana developers evaluate and approve the new release and try again. If it is still slower than you would like, take a look at the result set returned by your data sources (see the query inspector) and see if you can't make the query more efficient, that is, return fewer, ideally just one, row per label.

There are lots of new and improved features too, so it should be worth a look.

Regards, Michael Moore

On Fri, Feb 9, 2018 at 11:01 AM, Michael Moore mdmoore@gmail.com wrote:

I'm delighted you found it useful - and managed to setup the data source etc and get it working. Not sure about the slowness you talk about - I guess this might be a function of the amount of data you are processing, otherwise everything seems to respond instantly for me.

Maybe take a look at the query in the MySQL query client first and make sure it isn't taking long there.

The current version only supports 'table' type data sets (as opposed to 'time-series' - which I am working on right now), so somehow you have to make the mySQL query select a single value for each named metric - this might be a challenge depending on your ultimate data source. My idea in for this control imagined a set of metrics (such as stock tickers etc.) with values (prices?) changing at more or less random times. The query would then return the latest value of each of these metrics 'as of' some defined moment of time. Most likely, the current time, hence this means the latest value - still, I can imagine moving the query window back through history and hence setting the 'as-of' date/time as something other than the current latest values. Next up would be some sort of aggregation support, so the values might be some sort of average of the values existing during the query time window - this might get quite complicated though so it's all just an idea for now.

Anyway, generating the latest value for a set of randomly updating data points can be quite a complex task for MySQL, maybe this is your problem. If so, the time-series support I'm working on might come to the rescue, as the aggregation will then be done in the client. The downside will be that the result set returned by mySQL will have ALL the values, not just the latest values - so the network traffic will necessarily by much larger. Probably not an issue if the mySQL server is running on the same PC as Grafana.

On Sun, Jan 28, 2018 at 8:07 PM, karnamonkster notifications@github.com wrote:

Hi,

Firstly i would like to thank you for creating such an amazing panel for Grafana. This is what i was looking for a really long time. I did installed it with my installation and using it with the MySQL database. The browsing within the Edit Panel seem to be slow. Is there anything that we need to do before using the same to reduce the latency.

Thanks once again & Great Job !!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/michaeldmoore/michaeldmoore-multistat-panel/issues/2, or mute the thread https://github.com/notifications/unsubscribe-auth/ADjVrpA4UrlFC5N-SmhTXUwbHhY96b68ks5tPUP5gaJpZM4RwClE .

-- "That's not grey - I've just come from San Francisco with some flour in my hair"

-- "That's not grey - I've just come from San Francisco with some flour in my hair"

[karnamonkster]

Great Stuff.. Tried the new version 1.1.0 with my MySQL data, works like a charm.

[michaeldmoore]

I'm so glad it is working out.

My main client is asking about adding a second threshold level - a warning level in addition to the 'error' level threshold. I understand why they want this, but it's not all that obvious to me how it should be set up as a general feature - maybe have unlimited numbers of thresholds rather than just add on at a time etc.

Don't know if you have any thought on this.

The other thing I'm thinking about, basically when I implement support for timeseries queries, there will be a case where the bar itself displays a single value (generally, the last value for any particular time window, but this could be something else, such as the max or min, or average etc). Anyway, when we have data for a number of readings over time for each value, I'm thinking of how I could show this as a graph of some kind - either as part of the tool tip, or maybe as a semi-transparent spark chart overlayed on each bar.

Again, if you have any ideas, even if it is to say "this is a dumb idea", I appreciate the feedback.

Last thing, I know some people have trouble understanding what all the configuration options do or even where they would look for settings etc. If you run into any trouble and eventually figure it out - and can think of a better way of arranging or explaining everything, please speak up.

Regards, Michael Moore

On Sun, Apr 22, 2018 at 8:11 PM, karnamonkster notifications@github.com wrote:

Great Stuff.. Tried the new version 1.1.0 with my MySQL data, works like a charm.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/michaeldmoore/michaeldmoore-multistat-panel/issues/2#issuecomment-383442367, or mute the thread https://github.com/notifications/unsubscribe-auth/ADjVrhpfYBrFO_ZC4Q36f_VNYU9WJ7OAks5trUZhgaJpZM4RwClE .

-- "That's not grey - I've just come from San Francisco with some flour in my hair"

[michaeldmoore]

PS. did you find the "filter duplicates' setting? - I think this is the solution for the apparent slowness you reported earlier.

On Sun, Apr 22, 2018 at 9:08 PM, Michael Moore mdmoore@gmail.com wrote:

I'm so glad it is working out.

My main client is asking about adding a second threshold level - a warning level in addition to the 'error' level threshold. I understand why they want this, but it's not all that obvious to me how it should be set up as a general feature - maybe have unlimited numbers of thresholds rather than just add on at a time etc.

Don't know if you have any thought on this.

The other thing I'm thinking about, basically when I implement support for timeseries queries, there will be a case where the bar itself displays a single value (generally, the last value for any particular time window, but this could be something else, such as the max or min, or average etc). Anyway, when we have data for a number of readings over time for each value, I'm thinking of how I could show this as a graph of some kind - either as part of the tool tip, or maybe as a semi-transparent spark chart overlayed on each bar.

Again, if you have any ideas, even if it is to say "this is a dumb idea", I appreciate the feedback.

Last thing, I know some people have trouble understanding what all the configuration options do or even where they would look for settings etc. If you run into any trouble and eventually figure it out - and can think of a better way of arranging or explaining everything, please speak up.

Regards, Michael Moore

On Sun, Apr 22, 2018 at 8:11 PM, karnamonkster notifications@github.com wrote:

Great Stuff.. Tried the new version 1.1.0 with my MySQL data, works like a charm.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/michaeldmoore/michaeldmoore-multistat-panel/issues/2#issuecomment-383442367, or mute the thread https://github.com/notifications/unsubscribe-auth/ADjVrhpfYBrFO_ZC4Q36f_VNYU9WJ7OAks5trUZhgaJpZM4RwClE .

-- "That's not grey - I've just come from San Francisco with some flour in my hair"

-- "That's not grey - I've just come from San Francisco with some flour in my hair"

[karnamonkster]

You meant "Filter Multiples", Yes i did figured that out. Definitely the performance is better than before, I was also was trying to get some comments and feedback on your feature requests. (Will be dropping a note soon) Another thing i encountered, when i use a Join query for eg: select table1.*, table2.* from table1, table2 It does not give me all the rows, and only plots data from one Table

[pracplayopen]

unsure if i should reuse this issue, but had similiar experience.

unable to use this plugin because of slowness.

configuring even one raw metric pegged cpu and made browser tab unuseable.

ubuntu 16.04
chrome Version 67.0.3396.79 (Official Build) (64-bit)
grafana-server Version 5.0.4 (commit: 7dc36ae)
michaeldmoore-multistat-panel commit 620894e  (master)

[michaeldmoore]

After putting this on the back-burner for most of the year - I finally ran into the problem you and others have been complaining about. Chrome, apparently, was choking when there were more than a few dozen flashing items at the same time. And once choked, there was no way to un-choke it without reloading the browser. I don't know exactly why this was happening, but I've found a better way of managing the flashing items and now it is quite happy with several hundred items - enough for the biggest Christmas tree. I have quite a few new features too, related to ease of use and configuration as well as performance etc.

I'm putting the finishing touches to the documentation right now and should be re-submitting it to Grafana to complete the official release. Wish me luck.

Sorry for not responding earlier with what turned out to be a pretty straightforward fix. I hope you haven't lost heart.

Regards, Michael Moore

[michaeldmoore]

I appologize - this took much, much longer to address than I expected. My excuse is that I've been working other - paid - work, and you know how that works.

Also, I had never run into the performance issue that you and several others had observed. I thought it was due to some monster data queries overloading the system etc. But no - the issue was caused when more than a dozen or so bars were flashing, overloading the browser's capacity to render changes and locking up.

I found a different way to create the flashing effect and now it can flash hundreds of rows without problems. Phew.

I have an updated version of this ready to go - checked in to the 'verticalgroups' branch - soon to be merged (I hope) with the master branch - I'm just waiting for approval from the Grafana code review which is taking forever.

If you can, please try this and let me know how it works for you. This new version has a ton of new features and problem fixes too. Much more ready for prime-time.

[michaeldmoore]

Closing