Loom compatibility

[cornerman]

Wanted to check the status of this driver running with Loom (https://openjdk.org/projects/loom/).

As far as I understand, jdbc can generally benefit from running on virtual threads instead of actually blocking threads. So, I thought this would be useful for this jdbc driver, but I have not found any discussion about loom or virtual threads in this repo.

There seem to be two things to look out for though, because they can lead to still blocking threads with loom. Taken from the openjdk docs on virtual threads (https://openjdk.org/jeps/425):

There are two scenarios in which a virtual thread cannot be unmounted during blocking operations because it is pinned to its carrier:

    When it executes code inside a synchronized block or method, or
    When it executes a native method or a foreign function.

Pinning does not make an application incorrect, but it might hinder its scalability. If a virtual thread performs a blocking operation such as I/O or BlockingQueue.take() while it is pinned, then its carrier and the underlying OS thread are blocked for the duration of the operation. Frequent pinning for long durations can harm the scalability of an application by capturing carriers.

Doing a quick search in the repo for synchronized, I can see a few occurrences: https://github.com/search?q=repo%3Axerial%2Fsqlite-jdbc+synchronized+language%3AJava&type=code&l=Java

The question is probably, which one of the synchronized blocks or methods actually contain blocking code and whether we can change them to a ReentrantLock (as proposed).

Another question I have is: We obviously rely on JNI to call the sqlite3 C api and there is no way around it. So I am guessing, this will unavoidably lead to thread pinning. Does anyone know more about that?

Is your feature request related to a problem? Please describe.

The problem for me is currently to understand, whether using this jdbc driver with a virtual thread executor makes sense, and whether we can do things to improve support.

Describe the solution you'd like

I can see two results:

• documentation on usage with loom and whether it can have a benfit on performance
• check whether we can do something in the code to improve support

Describe alternatives you've considered

Additional context

Relevant efforts done in pgjdbc for postgres: https://github.com/pgjdbc/pgjdbc/issues/1951

[gotson]

My understanding was that synchronized blocks were not a problem anymore, though it was during the preview phase.

Another question I have is: We obviously rely on JNI to call the sqlite3 C api and there is no way around it. So I am guessing, this will unavoidably lead to thread pinning. Does anyone know more about that?

Yes, we use JNI for everything related to sqlite.

[headius]

The synchronized issue is being fixed in Loom updates, but I'm not sure if these are all being backported to 21 updates or not. It would probably be a good idea to try to move away from using synchronized in any case, since explicit locks will behave better on all Loom versions.

The JNI issue will be the long term challenge, to be sure. Unfortunately sqlite also does not appear to have any async APIs like other databases, or we could potentially use those to block outside of native code. As a result, Loom support would be limited: any blocking calls down into sqlite will pin the virtual thread to a native thread until the call returns.

I wonder, though... how many threads would you expect to have calling into the same sqlite database?

[gotson]

I wonder, though... how many threads would you expect to have calling into the same sqlite database?

for read, as many as you want. For write, if you don't use WAL, you will get busy timeouts if you use more than 1 concurrent writing thread. If you use WAL, that should work better.