Memory Leaking with latest release and dynamically filled shapes

[Selonie]

Firstly, thanks for the extension. Secondly, I've got a problem - I'm guessing it's my problem because I'm always doing things wrong. The thing is my little app has started to get rather greedy with the memory. For example – when dragging the points of a triangle the triangle is filled with a texture. The lines, points and texture that make up the triangle are cleared and updated as the points are moved. This was working correctly now it’s out of control. As you move the points the memory increases, it used to reach a max and the garbage would be collected – now it happens once if I am lucky and then the memory then just keeps on rising.

If I revert to an older swc the problem disappears. (How do I identify the version?)

The code below is a simple demo which exhibits this behaviour when using the latest swc. Perhaps someone could take a look and see if I’ve done anything wrong or see if it can be replicated. Many Thanks

import flash.geom.Point;
import starling.display.Image;
import starling.events.Touch;
import starling.events.TouchEvent;
import starling.events.TouchPhase;

import starling.display.Shape;
import starling.display.Sprite;
import starling.events.Event;
import starling.textures.Texture;

public class GraphicsAPIExample_Dynamic extends Sprite
{

    [Embed( source = "../assets/Checker.png" )]
    private var CheckerBMP      :Class;
    [Embed(source = "../assets/marble_80x80.png")]
    private var PinBMP          :Class;

    private var totalpins:int;
    private var pinArray:Array;
    private var pin:Image;
    private var pinRadius:int = 30;
    private var shapeDrawnWithLines:Shape;
    private var posX:int;
    private var posY:int;
    private var CheckerTexture:Texture;

    public function GraphicsAPIExample_Dynamic()
    {
        addEventListener(Event.ADDED_TO_STAGE, onAdded);
    }

    private function onAdded ( e:Event ):void
    {
        trace("touchHandler");
        pinArray = new Array();
        totalpins = 3;  
        var radius:int = 200;
        var location:Point = new Point(200, 200);
        var pinTexture:Texture = Texture.fromBitmap( new PinBMP(), false );
        CheckerTexture = Texture.fromBitmap( new CheckerBMP, false, false, 2 ); 
        for (var i:int = 0; i < totalpins; i++) 
        {   
            pin = new Image(pinTexture);
            addChild(pin);                              
            pinArray.push(pin);                     
            var angle:int = angle+(360/totalpins);
            pinArray[i].x = location.x + (Math.sin(angle*(Math.PI/180))*radius)/2;;
            pinArray[i].y = location.y + (Math.cos(angle*(Math.PI/180))*radius)/2;
            pinArray[i].addEventListener(TouchEvent.TOUCH, touchHandler);   
        }
        beginMaterialFill();
    }

    private function touchHandler(e:TouchEvent):void
    {   
        trace("touchHandler");
            var target:Image = e.target as Image;
            var touch:Touch = e.getTouch(parent);       
        if ( e.getTouch(parent, TouchPhase.BEGAN))
                {                                       
                    position = touch.getLocation(parent);
                    var posT:Point = touch.getLocation(target.parent.parent);
                    posX = target.x - position.x;
                    posY = target.y - position.y;
                }

        if ( e.getTouch(parent, TouchPhase.MOVED))
        {   
            var pin:Image = e.target as Image;
            touch = e.getTouch(parent);
            var position:Point = touch.getLocation(parent);
            pin.x = position.x - pinRadius ;
            pin.y = position.y - pinRadius ;    
            movin();
        }           
        if ( e.getTouch(parent, TouchPhase.ENDED))
        {                   
            position = touch.getLocation(parent);                                               
        }
    }

    private function beginMaterialFill():void //innitially fill the shape
    {   
        shapeDrawnWithLines = new Shape();
        shapeDrawnWithLines.graphics.lineStyle(4, 0xFF0000, 1);
        shapeDrawnWithLines.graphics.beginTextureFill(CheckerTexture);
        shapeDrawnWithLines.graphics.moveTo(pinArray[0].x + pinRadius, pinArray[0].y + pinRadius);
        for (var i:int = 1; i <= totalpins; i++)
        {
            if (i < totalpins)
            {
                shapeDrawnWithLines.graphics.lineTo(pinArray[i].x + pinRadius, pinArray[i].y + pinRadius);
            } else {
                shapeDrawnWithLines.graphics.lineTo(pinArray[0].x + pinRadius, pinArray[0].y + pinRadius);
                shapeDrawnWithLines.graphics.endFill();
            }
        }
            addChild(shapeDrawnWithLines);

    }
    private function movin():void 
    {           
        shapeDrawnWithLines.graphics.clear();
        shapeDrawnWithLines.graphics.lineStyle(4, 0xFF8040, 1);
        shapeDrawnWithLines.graphics.beginTextureFill(CheckerTexture);
        shapeDrawnWithLines.graphics.moveTo(pinArray[0].x + pinRadius, pinArray[0].y + pinRadius);
        for (var i:int = 1; i <= pinArray.length; i++)
        {       
        if (i <= pinArray.length-1)
        {                   
            shapeDrawnWithLines.graphics.lineTo(pinArray[i].x+pinRadius, pinArray[i].y+pinRadius);      
        }
        else    
        {                       
            shapeDrawnWithLines.graphics.lineTo(pinArray[0].x + pinRadius, pinArray[0].y + pinRadius);
            shapeDrawnWithLines.graphics.endFill(); 
        }
    }
    }
}

[jonathanrpace]

Thanks for posting the very helpful example - I'm able to reproduce the memory leak here too.

I've had a scan through the clear()/draw() cycle, and I can't see any obvious object references building up anywhere. Scout reports the memory is being gobbled by actionscript objects (not bitmaps or 'other'), and the GPU memory usage is nice and stable.

If anyone reading this has got a 'Premium' copy of FlashBuilder, and could run this example through the memory profiler, that'd help me loads (AFAIK Scout doesn't give a detailed memory breakdown) . I'm hoping a particular class type sticks out like a sore thumb with a gazillion instances of it being reported.

[jonathanrpace]

Ok - so this fixes around 90% of the leak. https://github.com/unwrong/Starling-Extension-Graphics/commit/7fb8463fbd984466fafb7d60c7557795a2ca201b

Memory usage still rises inexorably, though much more slowly.

Turns out not nulling private members during dispose is 'a bad thing'. I always figured that if the only reference to an object was owned by an orphaned object, then this child object should also be GC'd - ie, it's 'unreachable' by any other code.

Also of note, I was able to slow the memory usage further by nulling some more references in starling's DisplayObject.

[kerissakti]

Yes I'm also having this issue, it seems that any Graphics object such as Fill & Stroke are not disposed properly after the clear() function is called. Something somewhere is still referencing to the objects so they are never GC'ed. I'm using FB 4.6 AIR 3.6 ASC2

[Sudarmin]

Hi,I think I've fixed the issues, basically I modified the Graphics.as, Graphic.as, NGon.as, Fill.as, Stroke.as All in the graphics package if you want to try it, you can download it here : http://www.sendspace.com/file/0p82zd or http://www.2shared.com/file/fIq8YGam/starling.html http://www.2shared.com/file/WKZMJjBy/starling.html

[Selonie]

Sudarmin - I don't know what you've done but it seems to be working for me, memory drops back down to its initial level periodically. Thanks.

[jonathanrpace]

Hey Sudarmin - great job! - I grabbed your RAR earlier and found a few of the files weren't changed at all, and Fill added a 'Texture' public property which it then nulled during dispose.

So I'm not entirely sure how to merge this in (is it possible you've got an older version of the project checked out?).

It's possible I've messed something up somewhere - I'll take another look tonight. Or just send me over a pull request if that's easier for you.

[Sudarmin]

The most important part was in the "Graphic.as" in the dispose() function, someone forgot to remove the event listener, which made the GC unable to collect it, so I added this in the dispose function : Starling.current.removeEventListener(Event.CONTEXT3D_CREATE, onContextCreated);

I added the variable "texture" in graphic.as, because I saw when bitmapfill() was called in "Graphics.as", the texture that has been created from bitmapData did not get disposed properly (at least that is what I think), I forgot to submit the "Graphics.as", I modified a bit in there that made it more efficient, I think, I'll submit it tomorrow I don't have the file right now

[Sudarmin]

So here it is the complete Display package that I modified, might not be the best practice out there but I hope it can help
http://www.sendspace.com/file/uv6m2y http://www.2shared.com/file/dzh1TP-A/display.html

Thanks for creating this awesome extension though, cheers :)

[jonathanrpace]

Aaah! That'll explain why this leak wasn't occurring in older versions (I only recently added support for handling a lost context). It also explains why my 'unreachable' objects were in fact still reachable.

Thanks for the new download - looks good. I'll take a proper look tonight and get the final fix in there ASAP.

[jonathanrpace]

Ok, that's in. https://github.com/unwrong/Starling-Extension-Graphics/commit/f64b975f8710c75b034296f2fe474ff7278ffb56

Thanks again!

[wrench7]

There's still a memory leak somewhere in this extension. Not sure where at the moment, but hope to track down the culprit and will post when/if I find it.

[wrench7]

Ok, I've tracked down the issue to endFill() removing the _currentFill from the _container which then means that it's not disposed off properly when the Shape is also disposed of. Commenting out the code below in endFill() fixes the memory leak (though am not sure if there's any other ramifications to leaving the _currentFill on stage, there doesn't appear to be as far as I can tell)

if ( _currentFill && _currentFill.numVertices < 3 ) { _container.removeChild(_currentFill); }

[wrench7]

drawRoundedRect() without a fill being applied also leaks memory. drawRect without a fill doesn't however.

[jonathanrpace]

Rather than commenting out, doing this should fix the first issue.

if ( _currentFill && _currentFill.numVertices < 3 ) { _container.removeChild(_currentFill); _currentFill.dispose(); }

As for the second issue, I'd wager it's because RoundedRectangle... https://github.com/unwrong/Starling-Extension-Graphics/blob/master/extension/src/starling/display/graphics/RoundedRectangle.as

isn't overriding dispose() and null'ing its reference to 'strokePoints'.

I'll get a fix together tonight and let you know when it's in.

[wrench7]

Wow, that was quick! I actually tried your first fix, but in the opposite order (dispose, then remove) and it causes a #1009 error in the drawTriangles() method of the StandardMaterial Class context.setTextureAt( i, i < _textures.length ? _textures[i].base : null );

Same applies with the fix you've suggested so that may need to be looked into a bit more.

[jonathanrpace]

Thanks for the extra info. I've got a handle on what the problem is.

It boils down to me trying to be a bit too clever with sharing materials between different graphics (the aim being to avoid the situation where 100 drawn rectangles are being draw with 100 different shaders) - Because of the sharing, we can't dispose a graphic and its associated material without some sort of reference counting to ensure no other graphic is currently using the same material.

It's a tricky fix, so I'm going to give it some thought before I pile in. Once in, it should patch both these leaks.

[wrench7]

Hey Jonathan, any further thoughts on the memory leaks detailed above? Just a gentle nudge ;-)

[jonathanrpace]

Nudge acknowledged!

I'm afraid my day job is sucking up most of my time at the moment - so haven't really been able to support this project as much as I would like lately. Work will be calming down around October - I'l be back in full force then.

[IonSwitz]

Ah, shoot. Maybe this set of fixes will break this material sharing bit.

https://github.com/StarlingGraphics/Starling-Extension-Graphics/issues/58

Not having a good reproduction if the case with sharing materials, it's hard for me to determine if it is even possible to solve both problems here.

[jonathanrpace]

Leak fixed https://github.com/StarlingGraphics/Starling-Extension-Graphics/commit/853c9410d2bd3314678f979012e847acca7fc320

Note: This is quite a big refactor. Resource management is now much more sane, however this has come at the expense of beginBitmapFill() - use beginTextureFill() instead.

Allowing the API to create textures for you is just asking for abuse. I'm guessing everyone here is already very protective over how and when they create their Textures anyway, so won't mind this slight inconvenience.

[IonSwitz]

Nice work.

The one issue with removing the beginBitmapFill is the aim to have functionality that allows a direct import of graphicsData from flash readGraphicsData:

starlingShape.graphics.drawGraphicsData(flashShape.graphics.readGraphicsData());

But maybe that can be worked around, somehow turning a bitmapFill into a textureFill for this type of work.

I will grab your latest awesomeness and see what can be done to keep this otherwise pretty neat functionality (currently in the IonSwitz fork)

See https://github.com/StarlingGraphics/Starling-Extension-Graphics/issues/66 for further discussion, perhaps?

[LiuWong]

Memory leaks as before.

package test
{
    import starling.display.DisplayObjectContainer;
    import starling.display.Shape;
    import starling.display.Sprite;
    import starling.events.EnterFrameEvent;
    import starling.events.Event;
    import starling.textures.RenderTexture;
    import starling.utils.getNextPowerOfTwo;

    /**
     * ...
     * @author Liu Wong
     * 
     */

    public class testLeak extends Sprite
    {
        private const deltatime:int = 1;
        private var counter:int = 0;

        private var item:Shape;

        private var texture:RenderTexture;

        public function testLeak(prnt:DisplayObjectContainer):void
        {
            super();

            prnt.addChild(this);

            texture = new RenderTexture(128,128);

            addEventListener(EnterFrameEvent.ENTER_FRAME, onUpdate);
        }

        private function onUpdate(event:Event):void
        {
            counter++;

            if (counter >= deltatime)
            {
                counter = 0;

                if (item == null)
                {
                    item = new Shape();
                    addChild(item);
                }
                else
                {
                    var tmp:Shape = new Shape();
                    tmp.graphics.lineStyle(1, 0xff0000);
                    tmp.graphics.beginTextureFill(texture);
                    tmp.graphics.drawRect(0, 0, 32, 32);
                    tmp.graphics.endFill();
                    addChild(tmp);
                    tmp.x = Math.random() * 300;
                    tmp.y = Math.random() * 300;

                    item.graphics.clear();
                    item.removeFromParent();
                    item.dispose();

                    item = tmp;
                }
            }
        }
    }

}

[IonSwitz]

I can look into this, to find a solution here, but first I would like to know what it is that is leaking.

What objects don't get cleared in this scenario?

[LiuWong]

I can not say what it was, but the built-in Starling debug stats - showing the volume of used memory, demonstrates the rapid growth in this simple example.

[neolit123]

probably related to 6aa243be02c7? Graphics and Shape need proper dispose methods.

[IonSwitz]

I have checked in a dispose method on the Graphics class. The functionality can't be added to the regular "clear" method without hurting backwards compatibility, so I added a separate code path for this.

Please try it out.

instead of: item.graphics.clear() do item.graphics.dispose()

adding the Graphics.dispose() method to Shape.dispose() would unfortunately hurt backwards compatibility

I ran the sample for 5 minutes, and it kept returning to the same lowest level of memory usage after garbage collects

[neolit123]

thanks for looking into this,

given Shape is a top level class and it's sole purpose is to extend DisplayObjectContainer and encapsulate Graphics, i wonder what backwards compatibility it breaks to have a dispose() method in Shape that calls Graphics::dispose(). it just seems logical to have it...

if the BC issue is related to double "diposal" of sorts calls, this can be easily solved with a simple "isDisposed" flag and solve the compatibility in the mean time.

perhaps this should be reconsidered?

(edit: of course the idea would be to not force the user to explicitly call someShape.graphics.dispose() each time)

[IonSwitz]

The issue is that the behavior would change. In the current "clear" call, the Strokes aren't disposed, and it is not impossible to have a use case that would change it's behavior if the dispose call was included in the call to clear.

I would rather implement this in a way that makes the old behavior remain the same, and the dispose method for the explicit call. If I had more time to spend on the Extension, I agree that your approach is better, but I am unable to devote time to the project and therefore I prefer to make the risk lower that I break old implementations by changing the existing behavior.

Recreating shapes every frame is, in general, not recommended at all. The example with the drawRect is not that heavy, but replacing it with a drawCircle call makes the code a LOT slower.

[neolit123]

The issue is that the behavior would change. In the current "clear" call, the Strokes aren't disposed, and it is not impossible to have a use case that would change it's behavior if the dispose call was included in the call to clear.

can you better explain how exactly the backwards compatibility is affect by adding _graphics.dispose() in Shape::dispose()? are you suggesting the usecase where the user creates a Graphics instance, calls dispose() on it and then attempts to call other API methods such as clear() on the same instance, where the textures are already disposed of?

perhaps i can give it a try, but i'm finding trouble understanding the BC issue.

[IonSwitz]

The main point is that I don't have the time to investigate it, and I don't have time to work on patches if changing the dispose method breaks other behavior in already existing code using the SEG.

So I would prefer to use a separate method call to be on the safe side.

Writing an API is hard as hell, as concern has to be taken not just with internal consistencies but with the code that others have written that use these classes. For example, the extended Shape and Graphics classes, ShapeEx and GraphicsEx is just one example of code that might run into problems if the behavior of dispose() changes.

But, as I said, the main reason for not changing the current behavior is an extreme lack of time.

If you wish to investigate the consequences further, feel free. I would be very grateful.

[neolit123]

thanks for the answer,

So I would prefer to use a separate method call to be on the safe side.

i will take your overview as a good indication that the explicit graphics.dispose() is safer and "should suffice" at this point.

extreme lack of time

ditto on my end pretty much, which makes it very tempting not to investigate at all.