carlspring / s3fs-nio

A Java (NIO2) FileSystem Provider for Amazon AWS S3.
https://s3fs-nio.carlspring.org/
66 stars 23 forks source link

Apache MINA integration #844

Open SergiyBojko opened 3 weeks ago

SergiyBojko commented 3 weeks ago

Bug Description

I'm trying to expose Minio bucket through Apache MINA SFTP server (2.14.0) using example in the docs but it errors with:

 Not a directory: s3://Q3AM3UQ867SPQQA43P2F@play.min.io/foo/
    at org.apache.sshd.common.util.io.IoUtils.ensureDirectory(IoUtils.java:483) 

using following file system factory config:

        FileSystem fileSystem = FileSystems.newFileSystem(URI.create("s3://play.min.io/"),
                env, 
                Thread.currentThread()
                        .getContextClassLoader());
        Path bucketPath = fileSystem.getPath("/foo/"); // "foo", "foo/" and "/foo" does't work either
        return new VirtualFileSystemFactory(bucketPath);

Data from the file actually can be read by:

Files.readAllLines(fileSystem.getPath("/foo/bar")) 

I managed to bypass this by implementing custom FileSystemFactory, but after that MINA still fails because S3FileSystemProvider does not have newFileChannel method implemented.

This really doesn't seem right, am I doing something wrong?

Proposed Solution

Known Workarounds

Useful Links

Task Relationships

This bug:

carlspring commented 3 weeks ago

Hi @SergiyBojko ,

Would it be possible for you to provide a docker-compose script (or a test with TestContainers) with MinIO and/or a JUnit test case that we can try and reproduce this with?

SergiyBojko commented 2 weeks ago

Hi @carlspring Here is example project example.zip

You can run it with

./gradlew run

After server has started you should be able to run:

scp -P 2222 user@localhost:/bar abc.txt
passsword: p

this will try to copy file https://play.min.io/foo/bar and result in error:

14:17:50.617 [sshd-SftpSubsystem-64455-thread-1] DEBUG o.a.sshd.sftp.server.SftpSubsystem -- doSendStatus[ServerSessionImpl[user@/[0:0:0:0:0:0:0:1]:53170]][id=4,cmd=3] exception
java.lang.UnsupportedOperationException: null
        at java.base/java.nio.file.spi.FileSystemProvider.newFileChannel(FileSystemProvider.java:535)
        at org.apache.sshd.common.file.root.RootedFileSystemProvider.newFileChannel(RootedFileSystemProvider.java:193)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
        at org.apache.sshd.sftp.server.SftpFileSystemAccessor.openFile(SftpFileSystemAccessor.java:215)
        at org.apache.sshd.sftp.server.FileHandle.<init>(FileHandle.java:92)
        at org.apache.sshd.sftp.server.SftpSubsystem.doOpen(SftpSubsystem.java:967)
        at org.apache.sshd.sftp.server.AbstractSftpSubsystemHelper.doOpen(AbstractSftpSubsystemHelper.java:496)
        at org.apache.sshd.sftp.server.AbstractSftpSubsystemHelper.doProcess(AbstractSftpSubsystemHelper.java:353)
        at org.apache.sshd.sftp.server.SftpSubsystem.doProcess(SftpSubsystem.java:354)
        at org.apache.sshd.sftp.server.AbstractSftpSubsystemHelper.process(AbstractSftpSubsystemHelper.java:344)
        at org.apache.sshd.sftp.server.SftpSubsystem.run(SftpSubsystem.java:330)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
steve-todorov commented 2 days ago

@SergiyBojko We haven't had a chance to look into this yet. However, can you try the code you have with the 1.0.6-SNAPSHOT version which is published in Maven Central Snapshots? There was a bug related to the properties not being passed from the Files.newFileSystem down to the S3FileSystem which could have been related to the problem you are seeing.