Widgets not being updated after a period of time

[bashtoni]

When a browser has the dashboard page open for some time, some widgets fail to update. Not all widgets fail to update at the same time, but the longer the browser session runs, seemingly the more widgets fail. Typically the first one will fail to update after only a few minutes.

Setting Dashing.debugMode = true I can see the data is making it as far as the browser, but the widgets fail to update. Hitting refresh in the browser causes the page to load with the correct values.

I'm running dashing from 'dashing start'. Most widgets have data inserted via a curl request, although one is using a job. The widget using the job does not appear to exhibit the issue, although this is a graph widget - all others are numbers.

Any suggestions on how I can debug this further?

[florian-bayer]

We have the same problem with our dashboard. After a while all widgets do not update anymore, which is also visible in the updatedAtMessage. After a refresh of the page everything is up to date again.

[pushmatrix]

Are there any errors in the console? What browsers are you on?

[florian-bayer]

I use Google Chrome on Mac locally and we have a TVPeCee Android Strick running with Firefox (actually we wanted an Ricomagic MK802 but that wasn't available at that time). And both stop updating after a while. In the Chrome Console I can see the following: at the very beginning there is

Port error: Could not establish connection. Receiving end does not exist. miscellaneous_bindings:235
    chromeHidden.Port.dispatchOnDisconnect

And after something like every 1 to 5 minute there is this message:

Uncaught TypeError: Cannot call method 'filter' of undefined
    (anonymous function)
    stackData
    render
    Dashing.Graph.Graph.onData
    (anonymous function)
    Dashing.Widget.Widget.receiveData
    (anonymous function)
    (anonymous function)
Connection error
Connection opened

But at this point the widgets are still updating for one or more hours.

[bashtoni]

For us, there are no errors at the console at all - with debug mode on you can even see the new data coming in, but the data doesn't updated. Interestingly, the 'Last updated at' time continues to update.

[pushmatrix]

I have not been able to reproduce this issue. However, some googling of the error message seems to suggest it could be interference from a Chrome extension. Try disabling all your extensions, and see if it still freezes.

@florian-bayer: We tried using MK802s for our dashboards at Shopify, but those things ended up being crap. They would overheat all the time, and would boot only half the time.

[pushmatrix]

Two more questions:

• What type of server are you running Dashing on?
• Does this dashboard (http://dashingdemo.herokuapp.com/sample) freeze for you? Or is it only your own custom one.

[bashtoni]

Hi,

I don't see the issue on the demo dashboard.

On our dashboard, I only see the issue only on number widgets, both of which are updated once a minute from curl (but this may be a coincidence).

All other widgets are graphs updated from jobs, which never have a problem being updated.

The failing widgets are about as simple as they can be:

    <li data-row="1" data-col="1" data-sizex="1" data-sizey="1">
      <div data-id="newtickets" data-view="Number" data-moreinfo="Unowned, unreplied tickets" data-title="New Tickets" style="background-color:#96bf48;"></div>
      <i class="icon-envelope icon-background"></i>
    </li>

    <li data-row="1" data-col="1" data-sizex="1" data-sizey="1">
      <div data-id="pendingtickets" data-view="Number" data-title="Pending Tickets" style="background-color:#96bf48;" data-moreinfo="Tickets with customer reply"></div>
      <i class="icon-envelope icon-background"></i>
    </li>

[dieterdemeyer]

I'm having the same problem, running a custom dashboard on CentOS in Firefox. It only seems to be a problem with the Number and Meter widgets.

[pushmatrix]

@dieterdemeyer Are you also only using the API, or are you using jobs to push data to the widgets?

[dieterdemeyer]

@pushmatrix I'm using jobs to push data to the widgets. I also have the 'List' and 'Comments' widget on my dashboard and they keep updating.

[pushmatrix]

Ok I have an idea, go inside your number.coffee, and remove this line:

@accessor 'current', Dashing.AnimatedValue

Then tell me if it's still freezing up. I'm wondering if it's an issue with the number animation...

[dieterdemeyer]

Should i do the same within the Meter widget ?

@accessor 'value', Dashing.AnimatedValue

[pushmatrix]

No, leave that one as is. Then we'll see if that's the one that jams up.

[dieterdemeyer]

Actually, my current dashboard only has 2 Meter widgets, no Number widgets...

[pushmatrix]

How long till it freezes?

[dieterdemeyer]

Give or take close to one day...

[dieterdemeyer]

Currently testing with the Number widget and it seems to be not updating after a few minutes... Also combinations for payload data seem to be updating partially or not at all (i opened an issue for this..)

[bashtoni]

Removing the animation line from the widget seems to have done the job - typically for us it would stop working within 20-30 minutes, it's been running for four hours without a problem now.

Thanks

[florian-bayer]

Thats sounds good! We will try that as soon as possible on our dashboard.

[florian-bayer]

I just tested out dashboard in different browsers again (without the animate fix). In Chrome its running fine all day long, but in Firefox in stops updating after a while. So it might be a Firefox bug? It would explain why it is freezing on our Android TVPeCee Stick. There we use Firefox as well to display the dashboard (because it looks like it is the only browser on Android which is able to show real fullscreen). By the way, the TVPeCee Stick itself was running well, it was booting fast and didn't crash itself. But it just gives you 720p output, not as supposed 1080p, but it scales very good on a full HD Monitor so that itsn't that big problem.

[bashtoni]

We're using chromium running on a raspberry pi, and are no longer seeing the issue since disabling animation.

No problems with 1080p output either :)

[kevinthompson]

@bashtoni It sounds like your issues is solved, but I would recommend trying the Midori browser on Raspberry Pi instead of Chromium. Midori performs much better, and might actually be able to render the number animations without any issue.

We're running three Dashing dashboards on Raspberry Pi computers, each with a fair amount of animation, or frequently redrawn html (widgets with data-foreach), and we found that Midori (a webkit-based browser) outperformed Chromium on the RPi, specifically in regard to animation.

[pushmatrix]

@dieterdemeyer: This is still an issue for your right? Only on firefox?

[dieterdemeyer]

@pushmatrix yes, this is still an issue with Firefox.. I have to test my dashboard on Chrome..

[fydot]

I remove the line, but my dash still stop updating after about 20minutes?

[fydot]

the type of the wieghts stoping updating is Meter

[florian-bayer]

we don't have a meter on our dashboard and it is stopping as well, we have Numbers and Graphs and little own variations of those

[bstephenf]

I am also having the same issue and it is for both the Number and Meter widget. I have not yet tried removing the animation. - I notice these comments are from 2 months ago. Any recent updates?

[fydot]

no, this problem is still puzzling me.

[pushmatrix]

I'm going to look into this more this week.

[kevinthompson]

This has actually been a problem for us on our dashboards for a little while. As I mentioned in a previous comment, we're using the Midori browser running on Raspberry Pi computers using the Debian Wheezy distro.

After some period of time, the browser stops synching with the Dashing application. Refreshing the browser re-establishes the connection, so it seems that some browsers are simply failing to reattach to the event stream when losing connection.

I've been meaning to spend a little more time looking into this issue as well, but if nothing else, hopefully this information helps someone else dig into the issue.

[pushmatrix]

An ugly workaround for now would be a good ol' meta refresh <meta http-equiv="refresh" content="3600">

[myanaros]

I'm having the same issue, but involving a job that makes an http request to a rails application that responds with JSON data. When you initially load the dashboard the request goes through successfully, and the job continues to run and receive updated data from the Rails app but the widget doesn't show the updated value.

I've viewed the dashboard with Firefox on Ubuntu and Chrome on Windows 7, and have the Dashboard and Rails application running locally. I've also tried removing the line from number.coffee, but it had no effect.

Has anyone made anymore progress on this bug?

[Leviter]

We had the same issue on Raspberry Pi with Chromium, but after disabling the animation for the Number widget we can (at least) run it for 8+ (and counting) hours without any problem.

[Oshuma]

I'm having this same issue with Dashing 1.0.4 running on Sinatra 1.3.6. It seems to only affect the POST API, since Numbers and Meters get properly updated from jobs within the Dashing app, I just can't update them through the API (curl, etc.).

Edit: Disregard this. The issue was found to be elsewhere.

[joedeller]

I'm also seeing this using Firefox on windows, connecting to machines that are running dashing on Win7 x64, that receive data from the POST API. Refreshing the page does update, but not for long. Widgets are just lists and a graph. I also have dashing running on some Win 2K8 R2 servers and these don't appear to suffer from the same problem.

[mpavlov]

Also getting this on OSX running Chrome

[dblessing]

Also seeing this issue with some dashboards/tiles. Some tiles on the screen update dynamically while others do not. The ones that work and the ones that don't use different jobs, which could contribute to this. However I see nothing significant in the job to suggest why it wouldn't be updating. All "Updated at" timestamps show update time but the metric doesn't change.

[pushmatrix]

@mpavlov How long do you have it running before it craps out? Also, are you running the server on that same laptop, or on a diff server?

[mpavlov]

The jobs and the dashboard run on the same host. It takes anywhere between 3 mins and 30 mins for a widget to freeze, it appears random.

We have the jobs run every 30 sec, doing an http post for all widgets. Only meter and number widgets have the freezing problem, the others update reliably even when we leave the dashboard running for several days.

[derwin12]

Just a me too note .. using Chrome and exclusively number and list widgets.

[Dbzman]

I'm having a similar problem using the number widget. It looks like the issue only occurs when posting the same value twice. (which is a common case in our setup) After that the number doesn't change anymore. (Chrome and Firefox) I tried using another widget for displaying numbers (Hotness) which also changes its color. It has the same problem. The difference is that the color changes correctly but the value doesn't get updated.

Edit: Posting with a chrome extension results in "WARN -- : attack prevented by Rack::Protection::HttpOrigin". Posting different values results in a change but its stuck after posting the same value twice.

[mpavlov]

I checked the behavior described by @Dbzman by POSTing to a meter widget on a new dashboard generated by dashing 1.2.0.

Test 1:

curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "value": "3" }' http://localhost:3030/widgets/my_meter_widget
curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "value": "3" }' http://localhost:3030/widgets/my_meter_widget
curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "value": "4" }' http://localhost:3030/widgets/my_meter_widget

Results: Widget correctly updated to show "3", but not "4". It's stuck at "3" and does not update on further POSTs. A reload of the page correctly shows the last POSTed value.

However, changing value's type to int:

Test 2:

curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "value": 3 }' http://localhost:3030/widgets/my_meter_widget
curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "value": 3 }' http://localhost:3030/widgets/my_meter_widget
curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "value": 4 }' http://localhost:3030/widgets/my_meter_widget

Results: Everything works as expected and the widget does not get stuck.

Looks like the bug is either in the apparently stateful int-parsing, or in the widget's decision to attempt to display a string value in the first place.

[ericsolomon]

I am having a similar issue. After about 30 minutes my widget doesn't update anymore (with 'Last update at' time frozen) unless I restart the server. Right now I just have a cron job restarting it if it gets out of sync, but this is far from ideal. Has any progress been made on this? My widget is just a simple List widget that displays strings parsed from JSON objects.

[wickwire]

Hi, I think I have a solution for this:

I have a bash script processing some SNMP values, which in the end, echoes an integer.

I've been invoking this bash script in a job, to update a meter Synergy example widget. As with everyone else, the widget updates fine with the rand function but isn't doing so with the bash I have.

what I did on my sample.rb file, which seems to have solved it, is the following:

---

current_valuation = 0 current_karma = 0 current_synergy = 0

SCHEDULER.every '5s' do last_valuation = current_valuation last_karma = current_karma last_synergy = current_synergy current_valuation = rand(100) current_karma = rand(200000) current_synergy = (/home/wickwire/snmp.mem.calc.bash).to_i

send_event('valuation', { current: current_valuation, last: last_valuation }) send_event('karma', { current: current_karma, last: last_karma }) send_event('synergy', { value: current_synergy }) end

---

NOTE: current_synergy = (/home/wickwire/snmp.mem.calc.bash).to_i has the bash line inside ` accents, they're not showing on the post and I'm out of time to figure out how to escape chars)

I converted the bash output to int like stated above and left the meter.coffee untouched - which is to say, the line "@accessor 'value', Dashing.AnimatedValue" remained as it was.

Running dashing, the Synergy meter is updating with the animation working as well.

Before doing the cast, I also obtained the same results as stated in the thread: if I commented out the @accessor value line, I wouldn't get the animation, but the updates would work.

Please let me know if this fixes it for you as well!

[rafaelmagu]

I can confirm that using to_i on the variable before sending to the widget works. Been testing it without hiccups for the past two weeks.

Rafael Fonseca Sent from mobile

On 28/10/2013, at 2:57, wickwire notifications@github.com wrote:

Hi, I think I have a solution for this:

I have a bash script processing some SNMP values, which in the end, echoes an integer.

I've been invoking this bash script in a job, to update a meter Synergy example widget. As with everyone else, the widget updates fine with the rand function but isn't doing so with the bash I have.

what I did on my sample.rb file, which seems to have solved it, is the following:

current_valuation = 0 current_karma = 0 current_synergy = 0

SCHEDULER.every '5s' do last_valuation = current_valuation last_karma = current_karma last_synergy = current_synergy current_valuation = rand(100) current_karma = rand(200000) current_synergy = (/home/wickwire/snmp.mem.calc.bash).to_i

send_event('valuation', { current: current_valuation, last: last_valuation }) send_event('karma', { current: current_karma, last: last_karma }) send_event('synergy', { value: current_synergy }) end

I converted the bash output to int like stated above and left the meter.coffee untouched - which is to say, the line "@accessor 'value', Dashing.AnimatedValue" remained as it was.

Running dashing, the Synergy meter is updating with the animation working as well.

Before doing the cast, I also obtained the same results as stated in the thread: if I commented out the @accessor value line, I wouldn't get the animation, but the updates would work.

Please let me know if this fixes it for you as well!

— Reply to this email directly or view it on GitHub.

[mpavlov]

@wickwire to_i is another way of making sure the data sent to the widget is not a string. Unfortunately it doesn't seem to work for updates over http.

The right solution is for the widget to either reject strings (with an appropriate error message), or handle them properly. Since these are normally numbers-only widgets, the only case I can think of for wanting to handle strings is for when a value is not available, in which case displaying "n/a" or "unknown" makes more sense than -1.

[xfalcox]

I can confirm this bug happening on Number widget, updated via normal job.

Putting a to_i on the data sent through send_event method solved the issue.

[florianpilz]

Even though this is an old ticket, it's still current. We are using Chrome on Mac OSX and after 1-2 days the meter widget won't update the bar anymore. The number and Updated At text are still updated correctly, it's really just the meter bar.

A clean refresh reloads the data and everything is fine. But would be nice if the meter bar would always be accurate.

[mkb]

I was having this same issue and removing the quotes from the numbers in my curl commands fixed it, just as with the other folks.