Potential memory leak or GC bug

[ArcanoxDragon]

I have been working on a huge project using E2 (automated train system) and as the project has been progressing, I've noticed that I start getting lag spikes the longer the system runs.

Originally when I only had a couple chips (such as a linear motor controller and accelerometer), I didn't notice any lag during the time I was working. Now that I have probably 20+ chips (sound controllers, train logic controller, motor controller/acceleration monitor, EGP processor, etc) on the train, I start getting noticeable lag spikes after about 5 minutes of gameplay. The framerate will drop to 0 for a few milliseconds, and the spikes get longer the more I play

I wrote a quick STool to read the amount of memory used by Lua, and when it reaches about 45k is when the spikes become noticeable. If I reload the map or delete the chips it goes away. I haven't been able to find a good profiler so I can't do much further debugging myself, but I can update my project repository if anybody needs to profile the chips.

[ArcanoxDragon]

I just updated my repo anyways. Git url: https://bitbucket.org/briman0094/gmod-projects.git There is an "addons-extra" submodule that I can give authors access to if they have a Bitbucket account; this includes the addons required for the train to work. Let me know if you need any more info to debug the lag. The instructions for the project are in README.md in the root of the repository.

[TomyLobo]

would you use line breaks ffs?

anyway, gmod has some table with all the referenced objects somewhere iirc. look for that to see which object type gets stuck most

[ArcanoxDragon]

Line breaks where? I had word wrap on the README and the code should all have line breaks. I can print out the global table but it doesn't necessarily show what's causing the memory leak, it just shows what the leak was.

[TomyLobo]

I meant line breaks in your posts. And you should analyze the table, not print it out verbatim. Figuring out what is leaked might be helpful in finding out where it leaked.

Make sure it really is a leak though, first, by either computing some kind of unique identifier (table address?) or using a weak table (if gmod has that) and then filtering out the short-lived entries.

You could use __index metamethods and stack traces to find out where it's used From there, it might be possible to deduce where it was created :)

[ArcanoxDragon]

I've never quite understood how to use metamethods, but I think at least printing out the global table and all subtables would help determine which tables contain the most data.

[ArcanoxDragon]

I'm not seeing any huge tables or table contents, but it's getting about 0.25s lag spikes every 0.5 seconds. Any better way of profiling Lua, especially function calls? Since it's building up, I don't think it's short-term table entries unless the number of them being created is somehow climbing as time goes on. I feel like it's some internal data structure that's growing and a certain function call is taking longer than it should.

[Xandaros]

you can set a hook:

debug.sethook(someFunc, "c")

That will call someFunc on every function call.

[ArcanoxDragon]

Does it get called with any parameters? And is it called before or after the actual function is called?

[Xandaros]

Testscript:

debug.sethook( function ( a, b ) print( "Hook: " .. tostring( a ) .. ", " .. tostring( b ) ) end, "c" )
print( "Func" )
debug.sethook( nil )

Output:

Hook: call, nil
Func
Hook: call, nil

So... the callback doesn't have any noteworthy arguments and the hook fires before the call. (Which is usually how it goes, btw :D)

[ArcanoxDragon]

Hm. I can't see a good use for that in debugging this unless I could have a callback after each function call as well and get the name of the function call. If I had that info, I could build a table of call times.

[AbigailBuccaneer]

Hey look you can do exactly that and the code is literally given to you in the Programming in Lua book.

[ArcanoxDragon]

I'm a low-level coder, not a scripter, so I've never read it. Thanks for the link but a little less sass would be appreciated next time.

[AbigailBuccaneer]

i'm running at maximum sass, nobody can stop me now

[TomyLobo]

"I'm a low-level coder" he says as he dismisses Lua, the most barebones scripting language in common use, as being too high-level

[AbigailBuccaneer]

There's a difference between low-level and minimalistic. Lua is definitely not low-level.

[TomyLobo]

and then you can hook into all kinds of lua internals with lua itself and all that

[AbigailBuccaneer]

Sassiness issues aside, we should probably investigate this. :)

[syranide]

Without having checked anything too deeply, this seems like the old issue with Gmod and really the problem is the Lua GC. I don't remember the terms and all that, but the point is that the Lua GC has to pause the thread to garbage collect, as you create and/or store more objects it takes more time.

If you know your script does create lots and lots of objects (vectors, tables, etc), try to reuse them if possible or if otherwise possible, refactor MyList[0]["name"] to MyList["name"][0]. Thus causing significantly less objects to be created.

Addons is a common cause of this.

[TomyLobo]

you should probably explain how that helps and under what circumstances :)

[Nebual]

Anyone have any plans regarding profiling or performance? Should this be closed?

[AbigailBuccaneer]

I don't think anybody's planning on doing anything with this, and there's nothing to indicate that this is caused by Wiremod.