JuliaParallel / Dagger.jl

A framework for out-of-core and parallel execution
Other
621 stars 67 forks source link

`shard` executes f in sequence, not in parallel #473

Closed schlichtanders closed 5 months ago

schlichtanders commented 5 months ago

Just stumbling upon the internals of the shard function because I was looking for an easy way to distribute a function on all workers.

https://github.com/JuliaParallel/Dagger.jl/blob/cec17a14f75523bdcae3a6b00ede6c22d1d0fe9f/src/chunks.jl#L204

To me this looks like it should go into a separate for loop which directly follows thereafter, so that the different @spawn action can actually run in parallel.

jpsamaroo commented 5 months ago

Yep, good find!