datakernel server and its referenced objects not garbage collected after server shutdown and not having any strong reference

[KrishnaST]

datakernel server and its referenced objects not garbage collected after server shutdown and while not having any strong reference.

public class Main {

    static WeakReference<DataKernelServerService> reference;
    static WeakReference<DummyThnread> dummyReference;

    public static void main(String[] args) throws Exception {
        start();
        stop();
        for (int i = 0; i < 10; i++) {
            System.gc();
            System.out.println("kernel finalized : "+(reference.get() == null));
            System.out.println("dummy finalized : "+(dummyReference.get()  == null));
            Thread.sleep(2000);
        }
        Thread.sleep(50000000);
    }

    public static final void start() throws Exception {
        DataKernelServerService launcher = new DataKernelServerService();
        DummyThnread dummyThnread = new DummyThnread();
        reference = new WeakReference<DataKernelServerService>(launcher);
        dummyReference = new WeakReference<DummyThnread>(dummyThnread);
        new Thread(() -> {
            try {
                launcher.launch(new String[0]);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }).start();
        dummyThnread.start();
    }

    public static final void stop() {
        reference.get().shutdown();
    }
}

public class DummyThnread extends Thread {

    private int i = 0;

    public void run() {
        System.out.println("running dummy : " + i);
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    @SuppressWarnings("deprecation")
    @Override
    public void finalize() throws Throwable {
        System.err.println("DummyThnread : finalized");
        super.finalize();
    }
}
public abstract class HttpServerLauncher extends Launcher {

    public static final String PROPERTIES_FILE      = "http-server.properties";
    public static final String BUSINESS_MODULE_PROP = "businessLogicModule";

    @Inject protected PrimaryServer primaryServer;

    @Provides
    protected final Eventloop primaryEventloop(Config config) {
        return Eventloop.create().initialize(ofEventloop(config.getChild("eventloop.primary")));
    }

    @Provides
    @Worker
    protected final Eventloop workerEventloop(Config config, @Optional ThrottlingController throttlingController) {
        return Eventloop.create().initialize(ofEventloop(config.getChild("eventloop.worker")))
                .initialize(eventloop -> eventloop.withInspector(throttlingController));
    }

    @Provides
    protected final WorkerPool workerPool(WorkerPools workerPools, Config config) {
        return workerPools.createPool(config.get(ofInteger(), "workers", 4));
    }

    @Provides
    protected final PrimaryServer primaryServer(Eventloop primaryEventloop, WorkerPool.Instances<AsyncHttpServer> workerServers, Config config) {
        return PrimaryServer.create(primaryEventloop, workerServers.getList()).initialize(ofPrimaryServer(config.getChild("http")));
    }

    @Provides
    @Worker
    protected final AsyncHttpServer workerServer(Eventloop eventloop, AsyncServlet servlet, Config config) {
        return AsyncHttpServer.create(eventloop, servlet).initialize(ofHttpWorker(config.getChild("http")));
    }

    @Override
    protected final Module getModule() {
        return combine(ServiceGraphModule.create(), WorkerPoolModule.create(), JmxModule.create().initialize(ofGlobalEventloopStats()),
                ConfigModule.create().printEffectiveConfig().rebindImport(new Key<CompletionStage<Void>>() {
                }, new Key<CompletionStage<Void>>(OnStart.class) {
                }), getBusinessLogicModule());
    }

    protected Module getBusinessLogicModule() {
        return Module.empty();
    }
}
public final class DataKernelServerService extends HttpServerLauncher  {

    public static final int WORKERS = 4;

    public DataKernelServerService() {
    }

    @Override
    public void run() throws Exception {
        logger.info("HTTP Server is listening on " + Stream.concat(
                primaryServer.getListenAddresses().stream().map(address -> "http://" + ("0.0.0.0".equals(address.getHostName()) ? "localhost" : address.getHostName()) + (address.getPort() != 80 ? ":" + address.getPort() : "") + "/"),
                primaryServer.getSslListenAddresses().stream().map(address -> "https://" + ("0.0.0.0".equals(address.getHostName()) ? "localhost" : address.getHostName()) + (address.getPort() != 80 ? ":" + address.getPort() : "") + "/"))
                .collect(joining(" ")));
        awaitShutdown();
    }

    @Provides
    public final AsyncServlet servlet() {
        return RoutingServlet.create().map("/do", request -> HttpResponse.ofCode(200).withPlainText("called"));
    }

    @Provides
    protected final Config config() {
        return Config.create()
                .with("http.listenAddresses", Config.ofValue(ofInetSocketAddress(), new InetSocketAddress(8080)))
                .with("workers", "" + WORKERS);
    }

    @SuppressWarnings("deprecation")
    @Override
    public void finalize() throws Throwable {
        System.err.println("DataKernelServerService : finalized");
        super.finalize();
    }
}

Sample Output :

datakernel garbage collected : false dummy garbage collected : false running dummy : 0 DummyThnread : finalized datakernel garbage collected : false dummy garbage collected : true datakernel garbage collected : false dummy garbage collected : true

[necauqua]

Hello and thanks for your report.

We've found out that the issue is caused by both JmxModule module registering jmx-compatible components (the launcher itself and a server are both such compontents) to the global jmx registry while never unregistering them later and by the .awaitShutdown() and .shutdown() methods not being implemented appropriately for such use case.

While the latter one I've already fixed in b4c16180, the JmxModule is a bit harder to fix, so until it is fixed you can just remove it altogether - and also use the DataKernel branch that contains the fix I've mentioned.

[KrishnaST]

@necauqua Can the fix be expected in next release?

[eduard-vasinskyi]

Hello, @KrishnaST

We have fixed JmxModule related problem in 7905b1ca . Now all registered MBeans are unregistered upon launcher shutdown. This fix has not been yet released to maven central repository, but you can install a local version of datakernel to be used in your project.

For example, to install version 3.1 just run:

git clone --branch v3.1 https://github.com/softindex/datakernel
cd datakernel
./install.sh