When a single new BMS path is added in the launcher the directories under that path are not currently processed in parallel, resulting in slow (sometimes multiple hour long) process times. This uses a parallel stream instead of a regular stream to iterate over recursive directories and files under the added path instead of processing them sequentially.
This reopens #569, critically using the slightly faster parallelStream() interface of List over the slower stream.parallel().
I am aware that there had appeared to have been an issue resulting in the closure of #569:
I've tried to implement same code, but it sometimes causes OutOfMemoryError. (Maybe too much threads)
However after having multiple people test this change I cannot seem to get anyone to replicate the OutOfMemoryError exception. If @exch-bms2 encounters this again I would love to know if this wasn't a product of low available system memory, low JVM heap memory (under 4G) or a peculiar directory setup (symlinks, massive flat directory structure etc)
With this change users have reported massively reduced processing times (over 50x on some configurations)
When a single new BMS path is added in the launcher the directories under that path are not currently processed in parallel, resulting in slow (sometimes multiple hour long) process times. This uses a parallel stream instead of a regular stream to iterate over recursive directories and files under the added path instead of processing them sequentially.
This reopens #569, critically using the slightly faster parallelStream() interface of List over the slower stream.parallel().
I am aware that there had appeared to have been an issue resulting in the closure of #569:
Originally posted by @exch-bms2 in https://github.com/exch-bms2/beatoraja/issues/569#issuecomment-724024485
However after having multiple people test this change I cannot seem to get anyone to replicate the OutOfMemoryError exception. If @exch-bms2 encounters this again I would love to know if this wasn't a product of low available system memory, low JVM heap memory (under 4G) or a peculiar directory setup (symlinks, massive flat directory structure etc)
With this change users have reported massively reduced processing times (over 50x on some configurations)