Memory leak issue

[lafrank]

Hi There,

I would like to ask for some assistance how to cope with memory allocation when using IronPython from my C# application.

My code needs to compile scripts several times during execution in order to pick the right script from the list of available (registered with application) ones. In other words, I have a list of .py script files, and I need to find out which one is suitable to handle a certain situation.

Lately I realized that a huge amount of memory (-1.5 GB) is wasted and remains on managed heap. Using VS memory usage monitoring I found that the majority of allocated memory belongs to IronPython. And I can't find the way to reclaim that memory or prevent memory leak.

When compiling a script, what my code does is like below:

ScriptEngine PyEngine = Python.CreateEngine(options);  
ScriptSource src = PyEngine.CreateScriptSourceFromString(scriptSource, SourceCodeKind.Statements);
CompiledCode PyCode = src.Compile();
dynamic PyRuntimeCode = GetDynamicProperty(PyCode, "_code");
PythonAst PyAst = (PythonAst)GetDynamicProperty(PyRuntimeCode, "Ast");
ScriptScope newScope = PyEngine.GetScope("")

The above piece of code is inside an object that holds references to PyAst, PyCode, etc..., and this object implements IDisposable. Normally, I would call Dispose() on each of these objects or would use "using" term to ensure resources reclaimed after I don't need an object anymore, but I found that none of the objects IronPython or Microsoft.Scripting namespace implements IDisposable. There is no ScriptEngine.Dispose() for instance. So I am in a big trouble now how deallocate these resources.

And this is what left on heap after execution

Can someone please help me to understand how should I free the resources allocated by IronPython and ScriptEngine ?

Thank you.

[SteveUM]

I have seen somewhat similar issues. See my issues #278 and #241. I believe (but not sure) that you should create a single instance of the python engine and reuse it for your application. You can then swap out different scopes during your app session. Also, 2.7.8 has had some work done to improve memory allocation, but I have not had a chance to test yet.

[lafrank]

Thank you Steve for these suggestions. I am not sure that using a single python engine for multiple threads is a safe approach, but will definitely give a try to v2.7.8. I also identified a potential problem in the source code and fixed it, see my pull request,

[SteveUM]

Lazlo. I found this a while back on the IPy mail list:

"As long as you have a ScriptScope per thread, you should be able to share the ScriptEngine & Runtime between threads."

Not sure how accurate it is, but it was from Jeff Hardy who I think knew IPy very well.

[lafrank]

Thanks Steve, that sounds good to me. I will try this approach, too.