search

TanTayBui / flot

Automatically exported from code.google.com/p/flot
MIT License
0 stars 0 forks source link

Memory leak when redrawing with $.plot #269

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
I have an application that does real time charting.  I call flot, once a 
second with updated data. Basically I push new data onto the plotData 
Series array, and shift the data off the other end, so the number of 
points is always the same.  The display looks great, but there is a 
serious memory leak with IE8.  The data series has 700 points in it and I 
am leaking about 1.1MB/sec.  

Here is the code snippet that is the callback function from the 1sec timer 
that triggers the update from the server:

$.getJSON(getLiveData.php','tzo='+tzo,function(json) {
         plotdata[0].data.shift();
         plotdata[0].data.push(json);

       // This causes memory leak in IE

         $.plot($(placeHolder),plotdata, options );

                });

If I comment out the $.plot call, the leak goes away.  This means that the 
leak is some where within flot or excanvas. I have tried killing the 
placeHolder <div> and recreating it with jquery before the $.plot call, 
but that had no effect.  This works fine with firefox, so I am suspecting 
that the leak is in excanvas.  Is there something that I can call to 
dereference the memory that excanvas must be using before replotting?  
Right now I am stuck and am not sure what to do.  At >1MB/sec memory leak, 
it does not take long for IE to blow up.

Original issue reported on code.google.com by bensprac...@gmail.com on 3 Dec 2009 at 4:48

GoogleCodeExporter commented 9 years ago 
I extracted a test case that has the minimum amount of code to display the 
problem. 
I even have included links to the AJAX server files to get the live data in the 
hope 
that someone will look into this.  

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
    <head>
        <title>Flot Test Graph</title>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        <!--[if IE]><script language="javascript" type="text/javascript" 
src="lib/excanvas.min.js"></script><![endif]-->
        <script type="text/javascript" src="lib/jquery-1.3.2.min.js"></script>
        <script type="text/javascript" src="lib/jquery.flot.min.js"></script>
        <script type="text/javascript">
            $(document).ready(function() {                       
                var plotRendered =false;
                var plotdata;
                $.getJSON
('http://vbbtech.com/test/createTestChart.php','samples=700&freq=sec&',
                function(json) {
                    plotdata = [ {yaxis :1, data: json}];
                    $.plot($("#flottest"), plotdata );            
                    plotRendered = true;             
                }); 
                setInterval ( function() {
                    if (!plotRendered) return;
                    $.getJSON('http://vbbtech.com/test/getLiveData.php',function
(json) {
                        plotdata[0].data.shift();
                        plotdata[0].data.push(json);
                        // This causes memory leak in IE
                        $.plot($("#flottest"),  plotdata );
                    });
                },1000); 
            });
        </script>
    </head>
    <body>
        <div id ="flottest" style="width:800px; height:400px"></div>
    </body>
</html>

Original comment by bensprac...@gmail.com on 4 Dec 2009 at 12:22

GoogleCodeExporter commented 9 years ago 
I found an easy workaround for the problem.  Rather than call $.plot in the 
timer 
callback, I skip the initialization of the options, plugins and canvas and 
simply 
set the data, redo the grid and redraw:

        $(document).ready(function() {                       
                var plotRendered =false;
                var plotdata;
                var PLOT = null;  // flot plot object
                $.getJSON
('http://vbbtech.com/test/createTestChart.php','samples=700&freq=sec&',
                function(json) {
                    plotdata = [ {yaxis :1, data: json}];
                    PLOT = $.plot($("#flottest"), plotdata );            
                    plotRendered = true;             
                }); 
                setInterval ( function() {
                    if (!plotRendered) return;
                    $.getJSON('http://vbbtech.com/test/getLiveData.php',function
(json) {
                        plotdata[0].data.shift();
                        plotdata[0].data.push(json);
                        // This causes memory leak in IE
                        //$.plot($("#flottest"),  plotdata );
                        PLOT.setDate(plotdata);
                        PLOT.setupGrid();
                        PLOT.draw();
                    });
                },1000); 
            });

Original comment by bensprac...@gmail.com on 4 Dec 2009 at 4:01

GoogleCodeExporter commented 9 years ago 
Hi!

Thanks for the test case. I have not run it yet, but can you attach a dump of 
the
JSON to the bug so it won't go away when you change the server, I probably need 
a
local dump anyway because of the same origin policy (can't make Ajax requests to
other servers).

Original comment by olau%iol...@gtempaccount.com on 7 Dec 2009 at 4:32

GoogleCodeExporter commented 9 years ago 
I attaching a file with a snapshot of data from the server, per your request. 
the 
problem is that the leak happens when you feed it "real time" data.  You could 
simulate this without the AJAX request. The data is simply time vs a floating 
point 
value. You can see the format in the attached file.  New data samples are fed 
once a 
second by an Ajax call in the Timer call back, and an example of the data 
returned 
is "[1260189172000,4642.57]".

Original comment by bensprac...@gmail.com on 7 Dec 2009 at 5:34

Attachments:

GoogleCodeExporter commented 9 years ago 
I'm seeing a similar if not the same problem using xajax, not using json, i 
thought it was maybe xajax but the 
leak around the same amount as this.

$objResponse is the xajax object returned in php.

$objResponse -> script ( 'var d2 = ' . $abandon_str . ';var d1 = ' . 
$connect_str . ';
        var d3 = ' . $aHold_str . '; var d4 = ' . $cHold_str . '; var d5 = ' . $cDuration_str . ';
        $j.plot($j("#grafica-calls-day"),
            [{ label:"Llamadas Conectadas", data: d1, bars: { show:true, align: "center", barWidth:0.8}, shadowSize: 
0, stack:true},
             { label:"Llamadas Abandonadas", data: d2, bars: { show:true, align: "center", barWidth:0.8}, shadowSize: 
0, stack:true},
             { label:"En Espera Abandonadas", data: d3, lines: {show:true}, yaxis:2},
             { label:"En Espera Conectadas",data: d4, lines: {show:true}, yaxis:2},
             { label:"Duracion Conectadas", data: d5, lines: {show:true}, yaxis:2}]
            ,{
            legend: { show:true, noColumns:3, container: $j("#grafica-15min-legend") }
        },      
        grid: { hoverable: true },
        yaxis: { min: 0, tickDecimals:0 },
        y2axis: { ticks:[], min: 0, tickDecimals:0 }
});');

it seems as if a new object is created everytime $j.plot gets called but never 
gets released

Original comment by jsola...@gmail.com on 9 Dec 2009 at 5:19

GoogleCodeExporter commented 9 years ago 
Forgot to add, i experience the memory leak in every browser i've tried (IE, 
Firefox, Safai and Chrome)

Original comment by jsola...@gmail.com on 9 Dec 2009 at 5:22

GoogleCodeExporter commented 9 years ago 
Hi, can you post some working sample code? I am trying to do the exact same 
thing for
days but cant get my live update working?

Please? Thanks

Original comment by kerep...@gmail.com on 16 Dec 2009 at 10:51

GoogleCodeExporter commented 9 years ago 
I am not sure what happened, but since I originally made the post, the example 
which 
I included seems to have stopped working under IE, but works fine still under 
Firefox. Under IE the second call to $.plot is dieing within jquery.  I don't 
have 
time to debug this issue now.  I am using similar code with the workaround that 
I 
previously posted and it is working fine under both browsers.  The code 
attached to 
this issue does produce "live" charts.

I am convinced that the issue has nothing to do with the $.getJSON call and is 
simply that the some large object that is created within excanvas/flot is not 
getting released when a new call to $.plot is made.

Original comment by bensprac...@gmail.com on 16 Dec 2009 at 11:49

GoogleCodeExporter commented 9 years ago 
Issue 161 is probably related. There was some analysis recently on the mailing 
list:
http://groups.google.com/group/flot-graphs/browse_thread/thread/bc18c8e6f3eeff97
/36d2fa6646beb229

Original comment by olau%iol...@gtempaccount.com on 7 Apr 2010 at 4:52

GoogleCodeExporter commented 9 years ago 
Given the analysis in that thread, here's a first-pass patch at reusing the 
canvas 
elements when plot is called.

Original comment by ryl...@gmail.com on 9 Apr 2010 at 7:48

Attachments:

GoogleCodeExporter commented 9 years ago 
Hmm... found a bug in my patch, although can't seem to track down the issue.  
If you 
select something and then replot the same graph during the plotselected event, 
it seems 
to redraw the selection on the new graph.

I'm not familiar enough with the selection plugin to figure this one out yet, 
but I'll 
be looking into it...

Original comment by ryl...@gmail.com on 9 Apr 2010 at 10:31

GoogleCodeExporter commented 9 years ago 
I can confirm that calling setData, setupGrid and draw methods in a sequence, 
instead
of $.plot is very helpful. Before, each $.plot leaked over 10MB in Chrome for 
me, now
it's very minimal.

Original comment by mateusz....@gmail.com on 12 Apr 2010 at 11:19

GoogleCodeExporter commented 9 years ago 
In response to myself, just don't do that (replotting during a plotselected 
event) :)  
Otherwise patch seems stable to me.

Original comment by ryl...@gmail.com on 14 Apr 2010 at 6:32

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Hello,
any solution for this issue?
I'm having the same problem here and could not solve it.
Any tips?

Thanks

Original comment by muri...@gmail.com on 20 Sep 2010 at 12:47

GoogleCodeExporter commented 9 years ago 
First. reuse-canvas patch is working ok. I'm not sure if it fixes IE leaks, but 
it significantly lowers GC pressure on all browsers. It should be merged.

Second. As a workaround for IE I'm just reloading the page after some number of 
renderings. See here: 
http://github.com/membase/ns_server/blob/5aa20271557ac6778e9feb83e8c8e054a8f51d6
0/deps/menelaus/priv/js/analytics.js#L107

Original comment by alkondratenko on 18 Oct 2010 at 11:16

GoogleCodeExporter commented 9 years ago 
Hi people!

An update on this: I've just committed a patch based on issue 387 to wipe out 
the excanvas state. Hopefully that will fix IE from leaking.

Now as for the reuse-canvas patch: I am not really comfortable doing a hack 
like that without some proper thought. It seems it broke select-and-replot? 
What if somebody fooled around with the canvas state? We need to be sure the 
state is absolutely clear and that all the events are removed etc. to be able 
to do that in Flot, otherwise I'm going to be spammed by people complaining 
about code that used to work now failing in mysterious ways.

Also I am wondering what it buys us compared to just resetting the data with 
.setData(), .setupGrid() and .draw().

Another related issue is issue 387.

Original comment by olau%iol...@gtempaccount.com on 14 Dec 2010 at 9:12

GoogleCodeExporter commented 9 years ago 
Cool for IE.  I didn't notice any difference one way or the other though... 
With and without this patch I slowly leak.

I'm fine with dumping the reuse-canvas patch, its definitely hacky - although I 
don't really see the difference from what you've done in r284 for excanvas :)  
As for the spamming complaints, there are already lots of complaints about 
leaks from calling .plot over and over.  The docs suggest that you can do 
setData, setupGrid, and then draw, which is great (and doesn't leak)... and 
somewhat implies that it would be a good idea to do that instead of calling 
plot again. 

Would it be useful to have a public function like .replot or something that 
packages up the setData, setupGrid, and draw functions?  Might get people 
thinking instead of just blindly calling plot?

Original comment by ryl...@gmail.com on 14 Dec 2010 at 11:06

GoogleCodeExporter commented 9 years ago 
I would vote for a replot function that packs setData, setupGrid and draw. I 
experienced growing memory in FF and IE (not in Chrome) when calling $.plot 
every minute. It would get beyond 1Gb and then would make the browser unusable. 
I ended up switching to calling the three methods, which got rid of my problem. 
The memory still grows but stops at about 200Mb on my 4Gb machine and less than 
150Mb on a 2Gb machine. 

Having said that, there are still a few complaints (on the mailing list) that 
calling the three methods doesn't completely work. One person complained about 
the grid not rescaling properly and the other complained about the previous 
series still showing up.

http://groups.google.com/group/flot-graphs/browse_thread/thread/965d3b556748d67a
http://groups.google.com/group/flot-graphs/browse_thread/thread/5375fdd88f99d36b
/b826499ceb75191d

Original comment by mendesj...@gmail.com on 21 Jan 2011 at 12:18

GoogleCodeExporter commented 9 years ago 
Ryley: I agree the excanvas hack is ugly, but at least I can understand the 
consequences of it. Could you try unbinding events on as suggested by George 
here

http://groups.google.com/group/flot-graphs/browse_thread/thread/5375fdd88f99d36b
/b826499ceb75191d

and see if it helps your test case?

Otherwise, .replot(data) sounds fine to me. I'm not sure there were any good 
reason to split it up.

Original comment by olau%iol...@gtempaccount.com on 26 Jan 2011 at 3:45

GoogleCodeExporter commented 9 years ago 
Issue 387 has been merged into this issue.

Original comment by olau%iol...@gtempaccount.com on 11 Mar 2011 at 4:15

GoogleCodeExporter commented 9 years ago 
There's a helpful little test case in issue 387. I'm still hoping we can get 
this fixed so both $.plot and the .draw() ways of redrawing will work.

Original comment by olau%iol...@gtempaccount.com on 11 Mar 2011 at 4:26

GoogleCodeExporter commented 9 years ago 
Issue 386 has been merged into this issue.

Original comment by olau%iol...@gtempaccount.com on 14 Mar 2011 at 10:35

GoogleCodeExporter commented 9 years ago

Original comment by olau%iol...@gtempaccount.com on 14 Mar 2011 at 10:36

GoogleCodeExporter commented 9 years ago 
I spent most of yesterday and a great deal of the day today looking into this. 
I think my experiences mostly reinforce what people have been saying here on 
this bug and at other places. I started out with a slight modification of the 
simple example from issue 387 which shouldn't bind any events.

I found out the following things:

- IE appears to have been not too eager with releasing cleaned up DOM elements 
with jQuery 1.3.2, this also caused repeated calls to setupGrid() to cause 
leaks. Apparently jQuery 1.5 fixes this, I don't seem to get a leak with jQuery 
1.5.1, or at least it's very small. I tried running it through sIEve but sIEve 
appears to be unreliable for this purpose these days, at least I couldn't get a 
simple example from a jQuery bug report that sIEve claimed was leaking to 
actually lose any memory over time.

It also seems the old versions of jQuery were causing IE to leaking memory in 
Ajax call so if you were fetching data through Ajax, that would have been an 
additional source of memory consumption.

- Chromium is losing memory fast because it's not deallocating the canvases 
properly. Other Webkit browsers, at least on Linux, seem to be affected too. 
There's a bug report here:

http://code.google.com/p/chromium/issues/detail?id=69784

- Firefox 3.6 appears to be doing fine, although memory is jumping up and down.

In general, it seems the browsers are a bit sluggish in releasing removed DOM 
nodes. There are two sources of those in Flot, axis labels and the canvases. 
However, I'm planning to land a patch for using canvas text after 0.7 which 
would reduce it to the canvases.

With that in mind, and also the fact that all Flot examples are showing people 
to do the $.plot thing, I'm beginning to think hard about merging the 
canvas-reuse patch despite my earlier qualms. I think I can see 4 problems with 
it:

1. Surviving event handlers: but the built-in plugins are all registering 
events on eventHolder so the unbind in Ryley's patch should suffice.

2. Canvas size can vary over time: it turns out one can set the canvas 
width/height dynamically, it's in the canvas spec.

3. Surviving canvas state: perhaps doing a .save() immediately after creating 
the canvases, then do a restore() when reusing them.

4. Collisions from code still referencing the old canvas which would previously 
have been whacked but is now reused, I think the plotselecting problem falls in 
this category.

So of these, 4 is probably the main problem. I can't see any other way out of 
it than declaring the assumption that the old canvas is still alive buggy. For 
instance, in the selection event we could swap the emission of the event and 
the following redraw to avoid the assumption.

I have a preliminary revamped patch that does this, I'll see if I can finish it 
tomorrow and get it tested.

Original comment by olau%iol...@gtempaccount.com on 15 Mar 2011 at 8:25

GoogleCodeExporter commented 9 years ago 
I need to correct myself here, a) the legend is also making DOM objects and b) 
part of the problem in the selection plugin is that there are in fact still 
stray event handlers, in this case both one on the document and one in a 
setTimeout().

To solve b), the previous plot needs to know it has died. I'm currently 
experimenting with introducing an internal "plotshutdown" and a shutdown hook 
to enable plugins to unbind their events. Actually, I seem to have it working, 
but I think I found a bug in jQuery 1.5.1 event handling which makes it leak:

http://bugs.jquery.com/ticket/8545

Gah.

Original comment by olau%iol...@gtempaccount.com on 16 Mar 2011 at 8:32

GoogleCodeExporter commented 9 years ago 
I've committed this now. Had to drop the plotshutdown event because of the 
jQuery problem, instead I attach a plot data item to the placeholder and call 
.shutdown() on that.

If you find any problems with this, I'd like to hear, it's certainly possible I 
goofed up somewhere. :) For the time being, however, I'm going to assume this 
fixed the problems.

Thanks to everyone involved, both here, especially Ryley for the patch, and on 
other bugs.

Original comment by olau%iol...@gtempaccount.com on 17 Mar 2011 at 2:34

GoogleCodeExporter commented 9 years ago 
Holy moly! I was running into the same issue...I had several live graphs on the 
same page (and was actually using flashcanvas instead of excanvas). My memory 
was shooting through the roof after just a minute, about 1.2G! Eventually IE 
would die a horrible death. After updating to the latest it was like magic. 
Been on the page for several minutes and only using like 180M. It looks like 
there is still a small memory leak on my specific page. However, there are a 
couple other live components on the page which may be causing the issue. Either 
way, this is amazing. Thank you for fixing this!!!!!!!!

Original comment by bped...@gmail.com on 18 Mar 2011 at 8:04

GoogleCodeExporter commented 9 years ago 
Sadly it does not looks like this is completly fixed...
Using latest chrome or firefox 4 there is still a memory leak on live updating 
graphs, it is better than before but still there.

While trying to figure out wat happened I tried the chrome heap profiler and it 
reports an increase in HTMLDivElement, the leak doe not happen with the new 
realtime example so it may be the axis redraw, could it be the axis labels ?

Ps: the leak happen setData(), setupGrid(), draw() sequence.

Original comment by schmu...@gmail.com on 11 Apr 2011 at 9:20

GoogleCodeExporter commented 9 years ago 
What exact steps did you do to verify the code is leaking?

You need to reproduce this with a really simple test case, preferably one of 
the bundled examples. You can try modifying the realtime example to do a 
setupGrid call. The only reliable way of detecting a leak that I've found so 
far is repeating the plot calls a lot of times with timeouts (like the realtime 
example). Usually you get a sawtooth pattern with the browser increasing its 
memory usage for a while, then collecting garbage, then increasing again.

About the Chrome heap profiler - that doesn't necessarily mean a lot. sIEve for 
IE turned out to report false positives. 

By the way, just to clarify, there was no leak in Flot per se. The mentioned 
changes were updating jQuery to work-around IE problems and adding a 
work-around for (mostly) a Chrome bug with canvases.

Original comment by olau%iol...@gtempaccount.com on 13 Apr 2011 at 9:39

GoogleCodeExporter commented 9 years ago 
Thanks for the precisions, I think I jumped on a conclusion a little too 
quickly.
I ran a test last night by keeping my test application running for a long time 
and reducing the timer interval: the only thing observable is the sawtooth 
pattern you mentioned but the memory usage stay roughly the same.

Sorry for this :p

Original comment by schmu...@gmail.com on 13 Apr 2011 at 1:37

GoogleCodeExporter commented 9 years ago 
I still have a memory problem here, especially in Firefox. I am plotting time 
series data and redraw plots every ~30 secs or sooner; if I have lots of plots 
on the screen Firefox acts fine but grows sluggish with time, and memory usage 
shoots up to over a GB. The browser becomes very slow. Is there another known 
memory leak?

Original comment by delso...@gmail.com on 25 Jun 2012 at 11:35

GoogleCodeExporter commented 9 years ago 
"In general, it seems the browsers are a bit sluggish in releasing removed DOM 
nodes. There are two sources of those in Flot, axis labels and the canvases. 
However, I'm planning to land a patch for using canvas text after 0.7 which 
would reduce it to the canvases. I need to correct myself here, a) the legend 
is also making DOM objects"

I believe I am still seeing this, or a related issue. When I take a heap 
snapshot after I empty the plot's DOM element, I am seeing Detached DOM tree 
that contain HTMLDivElement for flot-tick-labels. I also see Detached DOM trees 
containing HTMLDivElement with className of "flot-base" and "flot-overlay". And 
others with className of "flot-text flot-y-axis".

There is definitely a leak.

Do I have to manually call shutdown or something in order to wipe out any 
existence of a flot instance from my page?

Original comment by mike.mal...@gmail.com on 9 Apr 2013 at 3:08