queryverse / Query.jl

Query almost anything in julia
Other
394 stars 49 forks source link

Support implicit multi-threading? #332

Open Moelf opened 2 years ago

Moelf commented 2 years ago

Since Query.jl is already not iterating but using getindex, is it possible to support multi-threading? Either baked in or via some externally passed in "executor" style like FLoops.jl

As an example to Query-style thingy with multi-threading: https://root.cern/doc/master/classROOT_1_1RDataFrame.html

In Julia we can match and surpass their speed: https://github.com/Moelf/UnROOT_RDataFrame_MiniBenchmark

And using Query.jl to mimic the style is almost overhead-free: https://github.com/Moelf/UnROOT_RDataFrame_MiniBenchmark/issues/1#issuecomment-914232278

So I'm wondering if there's a chance we also get multi-threading in the last quadrant of the thred-query 2x2 matrix

Single thread Multi-threading
For-loop
Query-style

(check means Julia performance match or faster than C++)