Closed malonzo47 closed 4 years ago
This may relate to other parallel processing issues. Feel free to close and merge. Basically: I'm hoping that if someone selects all of their processors, then we would see that most of their CPU capacity is being efficiently used.
Similar to #67. Closing.
Right now I believe that the block size passed to workers is large to reduce overhead, however, this seems to result in inefficient worker use, particularly for smaller datasets. Even for large datasets CPU usage is only about 1/2 of what I'd expect.