search

ocharles / weeder

A re-implementation of weeder using HIE files
BSD 3-Clause "New" or "Revised" License
169 stars 28 forks source link

Low parallelism usage #182

Open TeofilC opened 3 weeks ago

TeofilC commented 3 weeks ago

I've noticed that when passing -j8 to weeder, it often still uses only one core. See this graph from threadscope image

ocharles commented 3 weeks ago

There is unfortunately not much we can do to actually do the hard work in parallel, iirc. I don't know how to do a parallel reachability traversal.

TeofilC commented 3 weeks ago

I've dabbled with things like this before, so I might give it a go (when I have some time). After a quick look, I think I've spotted some other low-hanging performance fruit as well

ocharles commented 3 weeks ago

I look forward to seeing what you come up with!

ryndubei commented 3 weeks ago

I suppose the reachability traversal could be done more in parallel by searching graphs of individual modules first, taking exported declarations as roots. Then the final single-threaded search becomes a matter of searching through a graph of exports only, rather than all declarations. This may get more complicated in the presence of type class instances, though.