Release GIL in function generator

go-python / gopy

gopy generates a CPython extension module from a go package.

BSD 3-Clause "New" or "Revised" License

2.05k stars 113 forks source link

Release GIL in function generator #285

Closed pyalex closed 2 years ago

pyalex commented 2 years ago

When python callback is passed as an argument to a Go code, it's expected to work from any subsequently created go routine. Unfortunately, that's currently impossible due to GIL acquired and hold by the main go thread (in which Go was initially called).

This PR adds GIL release logic to the function generator. Before stepping into Go code, GIL will be released and then reacquired back.

Signed-off-by: Oleksii Moskalenko moskalenko.alexey@gmail.com

rcoreilly commented 2 years ago

This GIL stuff is out of my limited python expertise but I assume you've tested, and this is easily disabled if there are problems, so I'll merge. Does it impact performance?

pyalex commented 2 years ago

Hi @rcoreilly, as my previous PR this was as well tested within feast first. We have set of end-to-end tests that relies on this functionality and we also published at least two releases with this code and haven't heard complains so far. I'm not sure what you referring to regarding performance. GIL doesn't impact Go code in any way except blocking parallel Python code execution.

rcoreilly commented 2 years ago

Great! I was just wondering about the overhead of those save / restore calls which weren't otherwise being called..