neoforged / NeoForge

Neo Modding API for Minecraft, based on Forge
https://projects.neoforged.net/neoforged/neoforge
Other
1.03k stars 129 forks source link

Provide a Parallel Stream utility class #960

Open Shadows-of-Fire opened 1 month ago

Shadows-of-Fire commented 1 month ago

Java's parallelStream() method accesses the ForkJoinPool.commonPool when distributing the work, but this thread pool has the incorrect classloader when being used in an environment with Modules. This makes parallelStream() effectively unusuable.

There is a workaround, which is to execute parallelStream() inside of a ForkJoinPool task, which forces it to use the executing pool instead of falling back to the common pool. See https://stackoverflow.com/questions/21163108/custom-thread-pool-in-java-8-parallel-stream

We should provide a native ForkJoinPool with the correct classloader that can be used to execute parallel streams as a utility class, such as this one from FastSuite. This version is an extension of the original from Aequivaleo.

lukebemish commented 1 month ago

Given that a parallel stream executed inside a forkjoinpool task works just fine out of the box -- would it be possible to just run mod ctors in a forkjoinpool with the proper classloader and whatnot so that parallel streams works out of the box? I may be misunderstanding something about how parallel mod event dispatch works. The forkjoinpool seems to be set up in ModWorkManager -- would there be a way to make that set the thread context classloader to the relevant mod classloader when executing something?

Shadows-of-Fire commented 1 month ago

That would only be relevant for mod construction. If you attempt to use parallelStream() anywhere in your mod's code, you have to ensure that it is running inside a ForkJoinPool task with the correct context classloader, hence the need for the utility.

lukebemish commented 1 month ago

That's a good point, a helper is definitely needed then. If we could get it working out of the box in mod construction too that would awesome.

RaymondBlaze commented 3 weeks ago

The forkjoinpool seems to be set up in ModWorkManager -- would there be a way to make that set the thread context classloader to the relevant mod classloader when executing something?

The parallelStream workaround mentioned in the issue can not be apply to the mod construction. This would require a change on FML so mod construction tasks set the worker thread's context classloader to the correct one.

We should provide a native ForkJoinPool with the correct classloader that can be used to execute parallel streams as a utility class.

The native ForkJoinPool could be provided by FML and pass in its factory as a mod constructor parameter.

lukebemish commented 3 weeks ago

What shadows pointed out is that that is useful in mod construction but not during game run. Additionally, the workaround of running the parallel operation in a forkjoinpool task definitely does work during mod construction; I've actively used it, or at least my own equivalent, there without issue. Ideally I think you'd want to both provide a utility for this and make it work by default in the specific case of mod construction