search

ngageoint / geopackage-java

GeoPackage Java Library
http://ngageoint.github.io/geopackage-java
MIT License
81 stars 26 forks source link

Possible memory leak #43

Closed skocibusic closed 6 years ago

skocibusic commented 6 years ago

Please fill out as much known and relevant information as possible.

Version Information:

Expected Results:

Observed Results:

Output:

Relevant Code:

package test;

import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.List;
import java.util.Random;

import org.junit.Test;

import mil.nga.geopackage.GeoPackage;
import mil.nga.geopackage.manager.GeoPackageManager;

public class LoadTest {
    private static final Path BASE_DIR = Paths.get("D:\\Share\\GPKG");

    private static final List<String> GPKG =
            Arrays.asList([...]);

    private static final Random RNG = new Random();

    @Test
    public void loadTest() {
        for (int i = 0; i < 100_000; i++) {
            GeoPackage geoPackage =
                    GeoPackageManager.open(BASE_DIR.resolve(randomGeoPackage()).toFile());
            geoPackage.close();
        }
    }

    private String randomGeoPackage() {
        return GPKG.get(RNG.nextInt(GPKG.size()));
    }
}

Test Files:

Additional Information:

I added two screenshot, both from the same execution of the test. Maybe I oversaw something important, but I think beside opening and closing a GeoPackage there are no further steps necessary. Please correct me if I am wrong. And feel free to ask questions or request further details.

Cheers, Marko Skocibusic

image1 image2

bosborn commented 6 years ago

Thanks. I was able to reproduce the memory leak with other GeoPackages.

Try adding this call directly after the geoPackage.close() call: DaoManager.clearCache();

I need to investigate a little more, but it looks like new connection sources cause a new set of retained memory in the Dao Manager each time a GeoPackage is opened. If this is the fix, I'll add the call as part of the GeoPackage close and release it as part of 3.1.0.

I will be back next week and investigate further, but let me know if that resolves the memory leak on your side.

skocibusic commented 6 years ago

Hi Brian,

clearing the cache of the DaoManager does the trick. It fixes my test and is working in my production code. I will probably not call it everytime, but only once in a while.

Thank you very much for your fast feedback.

Cheers, Marko

Edit: By the way should I close this issue or wait with that until it is really fixed with version 3.1.0?

bosborn commented 6 years ago

Great, we can leave the issue open until a fix is released.

I'll have to see if there is way to clear the cache for just the closed GeoPackage (for when multiple are open). I didn't check yet to see what happens to a GeoPackage opened before the cache clear and used after.

bosborn commented 6 years ago

Upon closing a GeoPackage, added a call to clear the DaoManager from known and initialized ConnectionSource DAO types. Will release as part of 3.1.0.

Submitted an ORMLite issue and pull request to see if this can be improved upon in the future.

bosborn commented 6 years ago

Fixed in 3.1.0