IronLanguages / ironpython2

Implementation of the Python programming language for .NET Framework; built on top of the Dynamic Language Runtime (DLR).
http://ironpython.net
Apache License 2.0
1.07k stars 229 forks source link

Import not thread safe #661

Open tinyg opened 5 years ago

tinyg commented 5 years ago

Description

The import mechanism does not appear to be thread safe under heavy load.

Running the code:

var code = 
@"def method1():
    from System.Diagnostics import Stopwatch;
    from collections import Iterable";

var _engine = IronPython.Hosting.Python.CreateEngine();

var searchPaths = _engine.GetSearchPaths();
searchPaths.Add(@"C:\TFS\SDKS\ironpython\2.7.9\IronPython.StdLib.2.7.9.zip");
_engine.SetSearchPaths(searchPaths);

var runtime = _engine.Runtime;
var scope = _engine.CreateScope();
var source = _engine.CreateScriptSourceFromString(code, SourceCodeKind.AutoDetect);
source.Execute(scope);
var method = scope.GetVariable("method1");

Parallel.ForEach(Enumerable.Range(1, 1000), new ParallelOptions { MaxDegreeOfParallelism = 1000 }, (i) => method());

produces errors along the lines of "failed to import ...." or "error importing..." at least once in 1000 concurrent runs.

Steps to Reproduce

  1. Use LinqPad or add code into the main method of a console app
  2. Run
  3. Error is generated

Expected behavior:

Code runs to completion successfully

Actual behavior:

Error is thrown

Versions

IronPython 2.7.9 2.7.9.0 on .NET 4.0.30319.42000

tinyg commented 5 years ago

Workaround is to run this code (only once!)

import clr

def _():
    from System.Threading import Monitor

    import __builtin__

    __lock = object()
    base_import = __builtin__.__import__

    def new_import(name, globals={}, locals={}, fromlist=[], level=-1):
        Monitor.Enter(__lock)
        try:
            return base_import(name, globals, locals, fromlist, level)
        finally:
            Monitor.Exit(__lock)

    __builtin__.__import__ = new_import
_()
del _

def method1():
    from System.Diagnostics import Stopwatch
    from collections import Iterable
tinyg commented 5 years ago

Note that this code causes issues with using the clrtypes metaclass.

Suggested fix is to add a lock around the contents of import in Src\IronPython\Modules\Builtin.cs