search

marschall / memoryfilesystem

An in memory implementation of a JSR-203 file system
282 stars 36 forks source link

Support directory fsync hack on Linux and macOS #114

Closed marschall closed 5 years ago

marschall commented 5 years ago

In order to support Lucene on Linux and macOS with assertions enabled we need to support opening a FileChannel on a directory for reading on on Linux and macOS and supporting .force(boolean).

Reading from a FileChannel on a directory should still throw an exception. See #113

http://mail.openjdk.java.net/pipermail/nio-dev/2015-January/002979.html

https://bugs.openjdk.java.net/browse/JDK-8066915 https://bugs.openjdk.java.net/browse/JDK-8080629 https://issues.apache.org/jira/browse/LUCENE-6169

MichaelKunze commented 5 years ago

I have to come back to this. Thanks for taking time and fixing it. I'm using memoryfilesystem 2.1.0 now and i see the exception is gone if i'm using final var fileSystem = MemoryFileSystemBuilder.newLinux().build(); but if i use final var fileSystem = MemoryFileSystemBuilder.newEmpty().build(); with assertions enabled i still get the java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /index: is not a file

It's testable if you change it here: https://github.com/marschall/memoryfilesystem/blob/master/src/test/java/com/github/marschall/memoryfilesystem/PosixFileSystemExtension.java#L31

Is it suppose to do that?

marschall commented 5 years ago

Is it suppose to do that?

Yes, opening a FileChannel on a directory for read has the be explicitly enabled using MemoryFileSystemBuilder#setSupportFileChannelOnDirectory(boolean). Which is done by default in the #newLinux() and newMacOs() methods which give you a file system that behaves similar to Linux or Mac OS. You can not generally rely on the feature to be available on any operating system or even JVM. Even future versions of OpenJDK may not support the feature. Unfortunately I do not have a Windows machine available for testing but I believe the feature does not work on Window.

For your specific case I would recommend

MemoryFileSystemBuilder.newEmpty().setSupportFileChannelOnDirectory(true).build()