simonmar / monad-par

124 stars 37 forks source link

Add parallelism to Data.Vector while maintaining fusion #10

Open rrnewton opened 13 years ago

rrnewton commented 13 years ago

Related to Issue #9:

The monad-par package currently provides a "parMap" for everything traversable.

Sadly, this is woefully inefficient for data structures like arrays and Data.Vector, which can directly support a divide-and-conquer rather than element-wise traversal.

This raises the question of how best to add this functionality to vector. A separate vector-parallel package? Would that present any barriers to fusion laws (e.g. parallel maps fuse, and for that matter a serial + parallel map should perhaps fuse too).

rrnewton commented 13 years ago

Perhaps the right way to do this is to tweak Repa to use Monad Par. Currently Repa uses a heavy-weight strategy where it spawns a set of IO threads to do the work.

See forkGang: http://code.ouroborus.net/repa/repa-stable/repa/Data/Array/Repa/Internals/Gang.hs