search

Clemapfel / jluna

Julia Wrapper for C++ with Focus on Safety, Elegance, and Ease of Use
https://clemens-cords.com/jluna
MIT License
250 stars 13 forks source link

Julia and multi-threading #42

Closed vchuravy closed 1 year ago

vchuravy commented 1 year ago

I just came across https://clemens-cords.com/jluna/multi_threading.html and I wanted to provide a little bit of context. Julia uses thread-local-state on it's worker-threads and the GC needs to be able to find that TLS in order to scan the task stack. This causes your observation:

This is, because the Julia C-API is seemingly hardcoded to prevent any call of C-API functions from anywhere but master scope (the scope of the thread main is executed in).

The first thing many Julia functions do is to look at the TLS and use it to allocate, or interact with the runtime in other ways.

Julia 1.9 finally allows for foreign threads to be added https://github.com/JuliaLang/julia/pull/46609, hopefully this will simplify your work.

Clemapfel commented 1 year ago

Thank you! It is good to get a solid reason for the crash and I wasn't aware it is related to thread local storage.

I am aware of the 1.9 feature, I actually already wrote about it in the 1.0 version upgrade PR here a few days ago: https://github.com/Clemapfel/jluna/pull/39/files#diff-7b1e3ce989a2d943d61836597853e7dcb70f123646a6066545feba60faffa2faR56

But it is not merged yet so the current docs do not reflect this change.

I also marked the multi-threading docs as experimental and that the interface may be deprecated or removed completely, once 1.9 releases as stable I'll redo them but it will be be some time until then

Clemapfel commented 1 year ago

I may edit the docs with the context about TLS but I don't think any action regarding this issue needs to be taken so I'll close it. I do welcome the update though so thank you for the additional information