Allow plugging in ExecutorService

[fluentfuture]

As my company are finally getting ready to adopt virtual threads, we've been actively discussing what kind of structured concurrency we should use.

We can't wait for the JEP class. Plus, the API still looks quite crude from usability perspective.

We've also discussed the Mug StructuredConcurrency API, which is similar to your par() method.

Current thinking however, is that being able to plug in an ExecutorService is important to most of our servers, because besides the visible flow, our RPCs usually expect implicit contexts (such as security credentials, deadlines, trace info) to be passed through. A raw VT executor would not do any of that.

This context is historically accessed through some global state, and we have ExecutorService implementations that propagate the context. If we just take the JEP class, or Jox's par() method, or Mug's new StructuredConcurrency(), the context will be missing in the virtual threads, which is unacceptable. We'll also likely need to monitor virtual threads in production server in the near future.

We are currently playing with an idea to plug in the ExecutorService through ServiceLoader, so that server frameworks can have a way to control the Executor and do whatever crazy magic behind the scene.

And we are currently leaning against Mug's new StructuredConcurrency(executor) API approach because it's both error prone and verbose asking users to pass in the executor parameter. Instead, we might make the API looking more like Jox's static par() method, with a SPI backdoor for frameworks to plug in the executor (while users don't control the executor).

For example:

Robot makeRobot(...) throws InterruptedException, RpcException {
  try {
    return Fanout.concurrently(
        () -> tunnel(() -> fetchArm()),
        () -> tunnel(() -> fetchLeg()),
        (arm, leg) -> new Robot(arm, leg));
  } catch (TunnelException e) {
    throw e.rethrow(RpcException.class);
  }
}

If I didn't explain clearly, the tunnel() and TunnelException is our internal ErrorProne-checked utility to help propagating checked exceptions through lambda. I find it work nicely with structured concurrency, allowing us to still have the compile-time protection of checked exceptions, while side-stepping the whole ExecutionException awkwardness.

Ideas and feedbacks welcome!

[adamw]

Do I understand correctly that you'd like a global way to configure the executor service that is being used to create new virtual threads? Something like a static field, optionally initialized through a service loader? If so, sounds doable and without affecting the "default" way the library is used.

For the context propagation, though, maybe ScopedValues would be better suited? (although they are also only available as a preview)? Although I'm not entirely sure how you access the context via the ES currently?

[fluentfuture]

Huh. Sorry. It was mainly to start a dicussion and I was more interested in learning your thoughts and if you have any similar use cases, since VT adoption is still explorative for us.

[adamw]

Oh, no problem at all. We didn't yet have any reason to require custom executor services, but I'd be curious to learn about your design - and how you use ES to pass contextual information.

Observability for VTs is a challenge, starting with getting thread dumps. Though I don't think I have much to share yet in that area