fstat is failed on Java 17

[yshupletsov]

Moving to Java 17 I could not get fstat of the file descriptor:

java.lang.RuntimeException: native error calling fstat: Bad file descriptor -1
    at jnr.posix.util.DefaultPOSIXHandler.error(DefaultPOSIXHandler.java:24)
    at jnr.posix.LinuxPOSIX.fstat(LinuxPOSIX.java:117)
    at jnr.posix.CheckedPOSIX.fstat(CheckedPOSIX.java:135)
    at jnr.posix.LazyPOSIX.fstat(LazyPOSIX.java:132)
    at com.openwaygroup.appserver.monitoring.logprocessor.files.posix.PosixFileInformation.getFileInformation(PosixFileInformation.kt:15)
    at com.openwaygroup.appserver.monitoring.logprocessor.processor.EventProcessorImpl.processEvent(EventProcessorImpl.kt:36)
    at com.openwaygroup.appserver.monitoring.logprocessor.processor.EventProcessorImpl.processEvent(EventProcessorImpl.kt:19)
    at com.openwaygroup.appserver.monitoring.logprocessor.application.Application$startProcessing$1$1.invokeSuspend(Application.kt:34)
    at com.openwaygroup.appserver.monitoring.logprocessor.application.Application$startProcessing$1$1.invoke(Application.kt)
    at com.openwaygroup.appserver.monitoring.logprocessor.application.Application$startProcessing$1$1.invoke(Application.kt)
    at kotlinx.coroutines.flow.FlowKt__MergeKt$flatMapConcat$$inlined$map$1$2.emit(Emitters.kt:223)
    at kotlinx.coroutines.flow.SharedFlowImpl.collect$suspendImpl(SharedFlow.kt:372)
    at kotlinx.coroutines.flow.SharedFlowImpl$collect$1.invokeSuspend(SharedFlow.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)

The code:

...
val fileInputStream = FileInputStream(file)
val fileId = fileInfoSpec.getFileInformation(fileInputStream.fd) //fileInfoSpec is an interface, implemented for Windows and Posix (see below)
...
//Posix systems:
...
private val posix = POSIXFactory.getPOSIX()
...
override fun getFileInformation(fd: FileDescriptor): String? {
        try {
            val fstat = posix.fstat(fd)
...
//Windows:
    private val windowsPOSIX = POSIXFactory.getNativePOSIX() as WindowsPOSIX
    private val windowsLibC = windowsPOSIX.libc() as WindowsLibC
    private val serialNumberField = WindowsByHandleFileInformation::class.java.getDeclaredField("dwVolumeSerialNumber")
    private val fileIndexHighField = WindowsByHandleFileInformation::class.java.getDeclaredField("nFileIndexHigh")
    private val fileIndexLowField = WindowsByHandleFileInformation::class.java.getDeclaredField("nFileIndexLow")

    init {
        serialNumberField.isAccessible = true
        fileIndexHighField.isAccessible = true
        fileIndexLowField.isAccessible = true
    }
...
override fun getFileInformation(fd: FileDescriptor): String? {
        try {
            val handle = JavaLibCHelper.gethandle(fd)
            if (handle.isValid) {
                val byHandle = WindowsByHandleFileInformation(Runtime.getSystemRuntime())
                if (windowsLibC.GetFileInformationByHandle(handle, byHandle) > 0) {
                  val serialNumber = serialNumberField.get(windowsFileInformation) as Unsigned32
                  val lowIndex = fileIndexLowField.get(windowsFileInformation) as Unsigned32
                  val highIndex = fileIndexHighField.get(windowsFileInformation) as Unsigned32

Could it be related to the reflection warnings I got on Java 11?

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper$ReflectiveAccess (file:/C:/Users/yshupletsov/.gradle/caches/modules-2/files-2.1/com.github.jnr/jnr-posix/3.1.19/cc55c9e55541895498acdfd59c3145dff91db6e7/jnr-posix-3.1.19.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper$ReflectiveAccess
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Running the code with --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED has no success as well.

[yshupletsov]

Good day,

Any updates on this task?

[headius]

Sorry for the late response!

Yes, this is most likely due to the module restrictions and the warnings you received. You should be able to expose the necessary classes using the following JVM flag:

--add-opens java.base/sun.nio.ch=org.jnrproject.posix --add-opens java.base/java.io=org.jnrproject.posix

If that doesn't work post an example I can run and I'll help you get it going!

[yshupletsov]

Good day,

Looks like "--add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED" solving the problem.

[headius]

If you would rather not open those up to all unnamed modules, you can put the jnr-posix and other jnr jars in --module-path instead and use something like the line I provided above.

I would still very much like to see a reproducible example! I want to make a nicer error than the one the JDK produces and add some documentation for this. Can you provide a simple example that triggers the warning/error for you?

[yshupletsov]

The code should be like this with some jars in cp, but I cannot run it successfully on java8.

import jnr.posix.FileStat;
import jnr.posix.POSIX;

import java.io.File;
import java.io.FileInputStream;
import jnr.posix.POSIXFactory;

import static org.apache.commons.io.FileUtils.openInputStream;

public class PosixTest {

    public static void main(String[] args) throws Exception {
        FileInputStream fileInputStream = openInputStream(new File("test"));
        POSIX posix = POSIXFactory.getPOSIX();
        FileStat fstat = posix.fstat(fileInputStream.getFD());
    }
}

[headius]

@yshupletsov Thank you!

[headius]

Thank you for the example. I have pushed changes in #192 that will make it clearer what is necessary to avoid the error, as well as code to remove the warnings when running on Java versions earlier than 17.

This is the output from your example now, when no --add-opens flags are provided:

Some JDK modules may not be open to jnr-posix, which will break file descriptor and process APIs. See https://github.com/jnr/jnr-posix/wiki/Using-POSIX-with-Java-Modules
Exception in thread "main" java.lang.RuntimeException: native error calling fstat: Bad file descriptor -1
    at jnr.posix.util.DefaultPOSIXHandler.error(DefaultPOSIXHandler.java:24)
    at jnr.posix.BaseNativePOSIX.fstat(BaseNativePOSIX.java:139)
    at jnr.posix.CheckedPOSIX.fstat(CheckedPOSIX.java:135)
    at jnr.posix.LazyPOSIX.fstat(LazyPOSIX.java:132)
    at PosixTest.main(PosixTest.java:12)

See https://github.com/jnr/jnr-posix/wiki/Using-POSIX-with-Java-Modules.