sky-uk / cqlmigrate

Cassandra schema migration library
BSD 3-Clause "New" or "Revised" License
47 stars 29 forks source link

File Descriptor Resource not released on creating CqlPaths. #51

Closed chuckydev closed 7 years ago

chuckydev commented 7 years ago

On running cqlmigrate a new DirectoryStream object is created for each directory.

This is not closed, leading to the file descriptor resource being held for the duration of the run.

This is not so much an issue when cqlmigrate is being run once with the odd directory, but is a much larger issue when running cqlmigrate thousands of times. This results in the JVM's resources being filled up. We experienced this when running functional tests.

Current Implementation CqlPaths:

        directories.stream()
                .map(CqlPaths::directoryStreamFromPath)
                .flatMap(directoryStream -> StreamSupport.stream(directoryStream.spliterator(), false))
                .forEach(path -> addPathToMap(cqlPathsMap, path));

Proposed Solution:

    directories.stream()
                .forEach( directory -> {
                    try( DirectoryStream<Path> directoryStream = directoryStreamFromPath(directory)) {
                        StreamSupport.stream(directoryStream.spliterator(), false)
                                .forEach(path -> addPathToMap(cqlPathsMap, path));
                    } catch(IOException e){
                        throw new UncheckedIOException(e);
                    }
                });