search

MarkusJx / node-java-bridge

A bridge between Node.js and Java
https://markusjx.github.io/node-java-bridge/
MIT License
112 stars 6 forks source link

java.lang.OutOfMemoryError: Java heap space #98

Closed jacobshirley closed 6 months ago

jacobshirley commented 6 months ago

Hi there, first of all, thank you for this library. It really is the best node/java library available!

However, I have hit an issue related to memory that I don't know enough about how the JVM/Rust works to solve. As can be seen in the PR, i've created a test to show the error I'm getting when repeatedly allocating memory. It gets to about iteration ~500 and then comes out with java.lang.OutOfMemoryError: Java heap space. Perhaps there is a memory leak somewhere?

If it helps, I'm running Ubuntu 22.04

MarkusJx commented 6 months ago

I'll have to take a closer look but the first thing that comes to my mind is that we are working with two different garbage collectors at this point. You are creating a buffer inside node.js which gets passed to Java.

The node process keeps the resulting ByteArrayInputStream instance wrapper alive since this process only owns these light-weight wrappers, which are not creating a high enough memory usage, forcing the gc to run.

The Java process can only destroy the buffers if the node process releases the ByteArrayInputStream wrapper, which it doesn't.

You might want to try forcing the node gc to run, which may resolve your issue. Otherwhise I could look into providing a method for manually releasing Java objects.

But these are all guesses, I'll have to take a closer look, which may take a while.

jacobshirley commented 6 months ago

Thanks for the response. I suspect as you say, it's a garbage collection timing issue, where the memory is probably not getting released quick enough. I have just tried triggering the node garbage collector via global.gc() but no luck unfortunately.

A free type function which could release objects would be ideal though, without having to find ways around the node/Java garbage collection process.

MarkusJx commented 6 months ago

Funny thing, there was actually a memory leak. Will be fixed in #99.

Also added a new option to manually delete java references, which is still required when creating large objects inside the Java VM, since garbage collection is not guaranteed by the V8.

I'll close this merge request in favor of #99 since the JVM needs to be destroyed and re-created in order to set memory limits, which is not supported by this package due to memory safety issues.