search

input-output-hk / iodb

Multiversioned key-value database, especially useful for blockchain
Creative Commons Zero v1.0 Universal
95 stars 13 forks source link

LogStore file handles leak #21

Closed jankotek closed 6 years ago

jankotek commented 7 years ago

With many updates, LogStore might run out of available file handles. This manifests in LogStoreTest#concurrentKeyUpdate() test case:

java.lang.RuntimeException: java.nio.file.FileSystemException: /mnt/test/tmp/iodb0.578450809262877/store4058.journal: Too many open files

    at io.iohk.iodb.ForkExecutor.rethrow$1(ForkExecutor.scala:33)
    at io.iohk.iodb.ForkExecutor.finish(ForkExecutor.scala:38)
    at io.iohk.iodb.StoreTest.concurrent_key_update(StoreTest.scala:254)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Caused by: java.nio.file.FileSystemException: /mnt/test/tmp/iodb0.578450809262877/store4058.journal: Too many open files
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
    at java.nio.channels.FileChannel.open(FileChannel.java:287)
    at java.nio.channels.FileChannel.open(FileChannel.java:335)
    at io.iohk.iodb.LogStore.fileChannelOpen(LogStore.scala:842)
    at io.iohk.iodb.LogStore.fileOpen(LogStore.scala:835)
    at io.iohk.iodb.LogStore.startNewFile(LogStore.scala:270)
    at io.iohk.iodb.LogStore.compact(LogStore.scala:653)
    at io.iohk.iodb.LogStore.clean(LogStore.scala:697)
    at io.iohk.iodb.StoreTest.$anonfun$concurrent_key_update$4(StoreTest.scala:246)
    at io.iohk.iodb.ForkExecutor.$anonfun$execute$1(ForkExecutor.scala:21)
    at io.iohk.iodb.TestUtils$.$anonfun$runnable$1(TestUtils.scala:95)

We need to