PyCall Thread safety issues even with locking.

[the-noble-argon]

I'm trying to use PyCall in a background task while executing some pure Julia code, and I'm getting some thread safety issues. I tried putting all my python code into a ReentrantLock() but it's still failing. I made a relatively simple example of how to break this.

using PyCall
const PY_LOCK = ReentrantLock()
const PY_JSON = pyimport("json")

function write_json_file(fileName::String, outputData::Dict)

    lock(PY_LOCK)
    try
        open(fileName, "w") do outputFile
            PY_JSON.dump(deepcopy(outputData), outputFile)
        end
    finally
        unlock(PY_LOCK)
    end

    return nothing
end

function file_operation(fileName::String)
    outputData = Dict("a"=>randn(), "b"=>randn())

    write_json_file(fileName, outputData)
    return outputData
end

file_operation("testfile.json")
firstTask = Threads.@spawn log.(exp.(randn(10000000)))
taskList = [Threads.@spawn file_operation("testfile$(ii).json") for ii in 1:30]

The end result is an error as follows:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffa6e17001c -- PyDict_GetItem at C:\Users\user\Miniconda3\python39.dll (unknown line)
in expression starting at none:0
PyDict_GetItem at C:\Users\user\Miniconda3\python39.dll (unknown line)
PyObject_GetAttrString at C:\Users\user\Miniconda3\python39.dll (unknown line)
_getproperty at C:\Users\user\.julia\packages\PyCall\ygXW2\src\PyCall.jl:300
__getproperty at C:\Users\user\.julia\packages\PyCall\ygXW2\src\PyCall.jl:312 [inlined]
getproperty at C:\Users\user\.julia\packages\PyCall\ygXW2\src\PyCall.jl:318
write_json_file at g:\My Drive\julia_pycall_threadjson.jl:10
file_operation at g:\My Drive\julia_pycall_threadjson.jl:23
#4 at .\threadingconstructs.jl:258
unknown function (ip: 0000000061883543)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1838 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:931
Allocations: 10329372 (Pool: 10322778; Big: 6594); GC: 12

[the-noble-argon]

So more light has been shed on this and it may not be an issue with PyCall.jl

1. This looks like it's only a Windows issue (python39.dll showing up in the error message was a good hint). I was able to run this script repeatedly on linux without running into any issues.
2. It was suggested that I add the lines

   wait(firstTask)
   wait.(taskList)

   which appears to have resolved the issue on Windows (apparently Julia exits and tries to garbage-collect the results before the tasks are finished)