Causes firefox memory leak.

[bgood-trada]

I cycle through a few dashing dashboards with firefox or chrome on linux. Firefox slowly creeps up in memory usage over a day and will exhaust all memory on the system in one day. I have tried chrome and I get excessive bandwidth usage for the same period.

I don't see any special widgets being used. If I open firebug in firefox, one dashboard throws this exception (I think it's in number.html): "uncaught exception: series has no data {}"

Some numbers for an idea of how a dashboard causes a browser to misbehave- 1) firefox uses >4GB RAM and >4GB swap after running for 24 hours. 2) chrome downloads ~35GB/day of data.

Anyone know how I can debug to get a better idea of what the cause is?

[davefp]

Quick question: Can you provide a few more details about the platform you're using? Which linux distro? 32 or 64 bit?

For Chrome, you can see the network usage by popping open DevTool (Tools>Developer Tools) and selecting the Network tab. You'll probably have to refresh the page before it starts logging.

From there you'll be able to see each resource and when it was downloaded. The /events resource should keep growing, as it's where the streaming data is coming from. Over time you'll be able to see how much data is being downloaded.

On 12 August 2013 15:26, bgood-trada notifications@github.com wrote:

I cycle through a few dashing dashboards with firefox or chrome on linux. Firefox slowly creeps up in memory usage over a day and will exhaust all memory on the system in one day. I have tried chrome and I get excessive bandwidth usage for the same period.

I don't see any special widgets being used. If I open firebug in firefox, one dashboard throws this exception (I think it's in number.html): "uncaught exception: series has no data {}"

Some numbers for an idea of how a dashboard causes a browser to misbehave- 1) firefox uses >4GB RAM and >4GB swap after running for 24 hours. 2) chrome downloads ~35GB/day of data.

Anyone know how I can debug to get a better idea of what the cause is?

— Reply to this email directly or view it on GitHubhttps://github.com/Shopify/dashing/issues/187 .

[bgood-trada]

I've tried three different distros (all 64 bit). Linux Mint 15, Xubuntu 13.04, Slackware 14. I've tested at least Firefox 21-23 and probably a couple earlier iterations of firefox.

In chrome, the network counter climbs constantly on this entry: http://i.imgur.com/P0XKxhH.png Hovering over it shows: (anonymous function) @ application.js:13690 (anonymous function) @ application.js:13728 For each cycle through a dashboard, the content size of this entry climbs up to 380kb-500kb. I can't figure out where so much bandwidth is going.

In chrome the dashboard seems to initially load ~180kb and then the data downloaded slowly creeps up to that 380-500 range until the browser cycles to the next dashboard.

[davefp]

That data usage is way higher than it should be.

I just noticed that you say you're 'cycling through a few dashing dashboards' in the original issue. Can you expand on that a bit? It sounds like you may have written some custom code to do this, which might be causing the problem.

Also, what kind of data are you sending, and how frequently? Take a look at the jobs you're running on the server and see what they're sending. Maybe images are getting transmitted frequently, or you're updating a data-heavy widget with a high frequency.

[bgood-trada]

We're using sinatra_cyclist (http://www.rubydoc.info/gems/sinatra_cyclist/frames) to cycle through eight dashboards. If I try to open all eight dashboards at once to cycle through the tabs, only 5 dashboard pages open, the 6th opens partially, the 7th and 8th just hang. Tried in both chrome and firefox. None of the widgets seem to be data heavy.

[terraboops]

@pushmatrix and I are taking a look at the memory leak in the next little while. Memory leak aside...

This issue is ancient, so it deserves a worthy response! Seems like a two part question to me, with both parts relating to the operation of Dashing's Server Sent Event (SSE).

The first is a question that has been asked several times since this issue too: #387 #415 #582. The pages are hanging because each tab requires a persistent connection (SSE).

A little known fact about browsers is that you can only have a certain amount of connections open to the same host name at the same time. No matter what kind of connection! Firefox has a max of 6 (source: http://browserscope2.org/?category=network&v=top).

You can test this out yourself by creating a little web application that just does while(true);. Then make a bunch of AJAX (more than 6...) calls in a loop and watch the other calls sit in pending as the first 6 spin indefinitely. An SSE is effectively a while(true);-type persistent AJAX connection disguised by the EventSource object. That being said, this same problem exists for Web Sockets. Heck, the same problem exists even for the most versatile long-polling mechanism in existence: the Gif Socket.

There are workarounds:

• Traditional AJAX polling ($.getJSON().done(pollAgain)) masks this problem because each connection eventually gets it's turn in the queue after stalling for a period ... this may involve increasing the AJAX timeout values. :godmode:
• "Domain Sharding" - create a bunch of phony subdomains that all point to the same server (dashing1, dashing2, dashing3[.yourdomain.com]) ... This tricks the browser into allowing 6 connections per shard.
• "Domain Sharding" redux - use nginx or haproxy + virtual hosts as a front-end to each shard (could be the same box or multiple)
• Have multiple servers with different hostnames, split up your dashboards among them (max of 6 simultaneous per host)

The second issue cited is heavy data usage. This has also been asked in #439 - possibly others. Admittedly, Dashing is a bit of a heavyweight if you have several dashboards with lots of data-intensive widgets. This is because all events for all widgets on all dashboards are sent in the same HTTP endpoint (the endpoint that you pointed out - /events). This allows Dashing to start receiving data immediately, without having to parse the DOM before polling the SSE.

It should be possible to wait for the DOM to load, parse it and determine what data the current dashboard cares about and then selectively request the appropriate data from the SSE. This doesn't really fit in the 80% use case for Dashing though, so such a change should live as a Gem to extend dashing.