nagisa / rust_libloading

Bindings around the platform's dynamic library loading primitives with greatly improved memory safety.
https://docs.rs/libloading
ISC License
1.24k stars 102 forks source link

reload library cause memory leak #157

Closed tkkcc closed 2 months ago

tkkcc commented 2 months ago

Hi, i am trying to implement a plugin framework, that each plugin can update(via network) and reload itself without process restart. Currently, i fail to achieve this with libloading, because static variables in plugin are not freed on reload. I wonder if it is expected, and is there any solution(still using libloading).

To reproduce: The guest has 1GB static. The host keeps reloading guest. Then host memory usage keeps growing and gets killed finally.

plugin code:

use std::sync::LazyLock;

static BIG: LazyLock<Vec<u8>> = LazyLock::new(|| vec![0u8; 1000_000_000]);

#[no_mangle]
extern "C" fn my_func() -> u32 {
    let x = BIG[0];
    0
}

full demo: https://github.com/tkkcc/libloading_memory_leak_test

nagisa commented 2 months ago

Well, static implies a lifetime of 'static (i.e. extending to after the program is terminated), so they cannot be dropped, even if there were technical means to do so.

There are also other target-specific reasons why libraries might "fail" to unload cleanly. For example MacOS will not release a DSO if it has thread locals in it.

So if you require that memory is not leaked, using anything related to dlopen and friends is fundamentally incompatible with the requirements. Your best alternative would be to run the plugin as a separate process or rely on something like WebAssembly and wasmtime.