Support Virtual Thread Dispatcher for Pekko Projections

[JacobF7]

Currently, Pekko Projections only support the blocking-jdbc-dispatcher (see https://pekko.apache.org/docs/pekko-projection/current/jdbc.html). Are there any plans to start supporting a new dispatcher that uses Virtual Threads, similar to what has been achieved in the standard actor dispatcher ?

[pjfanning]

I don't understand why we would want to support a non-blocking dispatcher here. If you want, you can just change your own config to replace the thread-pool-executor with a virtual-thread-executor. I wouldn't recommend it but it in theory, it should work. Get back if there is some check that limits the type of executor that you can plugin.

[JacobF7]

Don't you think that since the Event Handlers need to persist into the database (blocking operation), we may benefit from using Virtual Threads since blocking on I/O doesn't block the OS Thread?

If we wanted to try using Virtual Thread Threads, do you suggest specifying our own dispatcher definition with the executor set to "virtual-thread-executor" ?

Edit: I've tried adding the above, but it doesn't seem to be working. Here goes a snippet of my conf:

    use-dispatcher = "pekko.projection.jdbc.virtual-thread-jdbc-dispatcher"

    virtual-thread-jdbc-dispatcher {
      type = Dispatcher
      executor = "virtual-thread-executor"
      virtual-thread-executor {
      }
      throughput = 100
    }

[pjfanning]

If we wanted to try using Virtual Thread Threads, do you suggest specifying our own dispatcher definition with the executor set to "virtual-thread-executor" ?

I answered this in https://github.com/apache/pekko-projection/issues/233

If you want, you can just change your own config to replace the thread-pool-executor with a virtual-thread-executor.

[JacobF7]

I've updated one of my comments above. But, in any case, here a snippet of my conf for projections. I've tried investigating what type of thread is executing inside the projection handler and it seems as though it's not a Virtual Thread.

  pekko.projection {
    jdbc.dialect = postgres-dialect

    use-dispatcher = "pekko.projection.jdbc.virtual-thread-jdbc-dispatcher"

    virtual-thread-jdbc-dispatcher {
      type = Dispatcher
      executor = "virtual-thread-executor"
      virtual-thread-executor {
      }
      throughput = 100
    }
  }

[pjfanning]

I had a quick look at the code and I can't see anything that would prevent this from working. There are no logs or reproducible test cases to back the assertion that it doesn't work. I may have time later in the week to try to test this myself.