Detect virtual threads and reduce memory needs

[StefanLobbenmeierObjego]

Describe the Enhancement

Virtual Threads require less memory (source). Now that spring boot supports them in 3.2, is it possible to detect that they are enabled and reduce thread memory in the build pack?

Possible Solution

There is already detection for reactive spring, which reduces the thread count:

If the application is a reactive web application
Configures $BPL_JVM_THREAD_COUNT to 50

Since virtual threads are configured in the application.properties, I am not sure if it is also possible or if the build pack just looks at dependencies.

Motivation

Less memory reserved for threads = more memory for the application, or cheaper hosting costs.

Also looking for input from others, because I am not sure how much memory this will save / How many threads are still there with this setting. For me the best way to test this would be to run the application once with virtual threads and once without and compare memory needs.

[dmikusa]

I really like this idea. It would be great if buildpacks could auto-detect and adapt to this difference in configuration. That would be a nice benefit of using buildpacks.

The way that we do this now for Reactive/Servlet/Non-web is to look at what classes are present in the application. If there were a way to determine if virtual threads were being used in this manner, that would be an option. Because virtual threads are part of the JVM and not a JAR that gets included with the app, I'm not sure this option would work. I'm not an expert on virtual threads either though, if someone knows of a way we could make this work let me know.

I'm not 100% sure how we might do that in a manageable way. Because of the complexity of application.properties/application.yml, attempting to read values out of those files is not something that I think we can reliably do. We at least try to steer clear of this approach. What we might be able to do is to look for system properties or environment variables that would indicate virtual threads are being used. That would not cover the case where someone enabled it directly in application.properties|yml, but if -Dspring.threads.virtual.enabled=true is set or if SPRING_THREADS_VIRTUAL_ENABLED=true, we could easily detect that, and adjust the configuration. It's not perfect, but may be better than nothing.

Another way that might be an option would be to flip things around and have the user declare to buildpacks that they want virtual threads enabled. Then the buildpack could set the corresponding env variable or system property to enable it and at the same time reconfigure the number of threads set. I'm not a huge fan of this approach though as I like to avoid introducing new buildpack-specific settings like this. It increases the complexity of buildpacks and what users have to know to make them work. I'm a much bigger fan of making things "just work" based off existing settings and options users already know.

I'm not also totally sure how users will want to configure virtual threads, and I think that kind of matters. If it's something users will want to enable across all environments (i.e. dev/test/prod) or if they might want to turn it on/off here and there. The former is probably easier done via application.properties, while the latter is better with external configuration. The way people use it might affect the approach we want to take as well.

Anyway, this is all me thinking out loud. I really like the suggestion, if folks have thoughts on the implementation feel free to post them here. We're listening.

[StefanLobbenmeierObjego]

Yeah I figured that parsing the application.yaml is probably a bit too difficult, especially considering that the files might be inactive depending on profile. So I agree with that it is probably easier to flip this around as you said. We want to roll out virtual threads in the coming weeks, so maybe I can also post my insights on how memory usage changes here.

[dmikusa]

We want to roll out virtual threads in the coming weeks, so maybe I can also post my insights on how memory usage changes here.

That would be fantastic! Thanks

[burl21]

Any updates?

[StefanLobbenmeierObjego]

Hey 😄 sorry I did not have that much time to look at it in the end, but I can still share some data. From our io.micrometer:micrometer-registry-prometheus data it seems there was little difference for our service memory usage, around 20MB in total jvm memory usage when I did some manual load testing.

But I suspect that I was doing something wrong and maybe there is better tooling to look at memory used / reserved for the stacks, because io.micrometer:micrometer-registry-prometheus only includes information about the number of threads, but nothing referring to thread stacks.