Closed techsy730 closed 3 years ago
Also, the supplier method gets called only on recursive steps, not inner loops. So this overhead of tracking will be negligible compared to the "real" work.
Just a heads up, I have a branch mostly fixing this, but IRL work got busy before I could clean it up for submission.
When using, for example,
IntArrayList.toListWithExpectedSize
in a parallel stream, every thread will pre-allocate the size given. While this doesn't break correctness, it is rather wasteful, especially if given a large size on a highly multithreaded system.Only one thread (the one that will merge it all into the final result) needs the full array. The second thread only needs ~half of the given size preallocated (as it will only be handling about half the input). The 3rd and 4th only need a ~quarter of the size preallocated. Etc
For the
toXWithExpectedSize
collectors may want to replace the supplier function (the one that allocates each threads' mutable accumulator) with a stateful one that tracks how times it has been called and reduce how much it preallocates as the calls go up (to some floor, like 8 or something). The "correct" way would bepreallocateSize / (floor(log_2(timesCalled)) + 1)
, but that may be too complicated to be worth it. A simpler method that isn't quite as optimal but still gives a meaningful improvement to the current state ispreallocateSize / timesCalled
.This assumes that the first thread is indeed the one that will merge together the final result. If it isn't, then all this gets thrown out the window and there is no safe way we can determine which threads get how much.
Alternatively, we can leave the methods as they are and just warn as an
@implNote
that these methods may be a bit wasteful with memory if used on parallel streams.