anskaffelser / vefa-validator

Generic validator as a Java library originally created for use within eProcurement.
17 stars 17 forks source link

Memory leak in validator? #35

Closed jphuttunen closed 7 years ago

jphuttunen commented 8 years ago

It seems that there may be some problem with memory handling in the latest Validator version.

When doing repetitive instantiate -> validate cycle for sufficiently many times, Java VM will exit with an OutOfMemory exception. If default garbage collection overhead limit is used, the program terminates after N validations (N depends on max heap allocated for JVM) with OutOfMemoryException "GC overhead limit exceeded". If GC overhead limit is disabled, the program terminates after N validation with OutOfMemoryException "Java heap space". The number of validations (N) can be very low, depending on the document and max memory allocated to JVM. In my test case, 512 MB max memory and 110k xml document brought out the problem in 10 repetitions, while increasing the available max mem to 6 GB allowed about 160 repetitions before crashing.

Is there a way to circumnavigate or alleviate this problem? Using only one validator instance could be one solution, but i haven't tested whether it would work in the long run either (and it would probably not be the nicest solution to the problem).

Minimal program to reproduce the error (uses vefa validator 2.0.2 and latest production artefacts from 17th November 2016):

public class App 
{
    public static void main( String[] args )
    {
        String documentFilename = "DOCUMENT_TO_VALIDATE.XML";
        String artefactPath = "PATH_TO_ARTEFACTS";
        int repetitions = 200;

        App app = new App();
        try {
            for (int i = 0; i < repetitions; i++) {
                System.out.println("Running iteration " + i);
                app.validationOperation(documentFilename, artefactPath);
            }
        } catch (Exception e) {
            System.out.println(e);
        }
        System.out.println("Finished");
    }

    private void validationOperation(String documentFilename, String artefactPath) throws Exception {
        no.difi.vefa.validator.api.Validation validation;
        Source source = new DirectorySource(Paths.get(artefactPath));
        Validator validator = ValidatorBuilder
                .newValidator()
                .setSource(source)
                .build();
        validation = validator.validate(Paths.get(documentFilename));
        validator.close();
        validator = null;
        validation = null;
        source = null;
    }
}
klakegg commented 7 years ago

The validator is not supposed to be reinitiated for every validation. Tracking down bugs related to use not according to intentions has no priority.