Goofys needs support for mknod to work in NFS client environments..

[janibashamd]

Hi Team,

Please check the issue rasied in below URL - https://github.com/kahing/goofys/issues/171#issuecomment-294687949

Steps performed -

1. Installed swift3 in CentOs
2. Installed goofys and mounted swift container locally
3. Able to access container without issues.
4. Then installed NFS server in same system which mounted swift container
5. Installed NFS client in one of network systems
6. In NFS server gave swift container mounted location in NFS exports file
7. Did exportfs -a
8. On the NFS side mounted server side swift container mounted location locally in NFS client.
9. Mount was successful and can read/list objects but when i try to create/edit files getting permission issue. It says read-only permissions.

As per above bug, issue is mainly because of mknod. So requesting goofys support for mknod..

[kahing]

I've come to realize that NFS support involves much more than mknod (which is relatively easy). How are you guys planning to use NFS? (v2, v3, or v4, and what type of workload?)

[janibashamd]

Currently we are running on NFS V3 and may be moving to NFS V4 in the future for user authorization. We just need good performance on NFS client side with read & write permissions...

[kahing]

Why do you need NFS instead of mounting the buckets on the clients? On May 11, 2017 22:20, Mohammed Janibasha notifications@github.com wrote:Currently we are running on NFS V3 and may be moving to NFS V4 in the future for user authorization. We just need good performance on NFS client side with read & write permissions...

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

[janibashamd]

So that users can't know back-end is swift and we don't want to give swift user authorization for each user..

[kahing]

This is unlikely to work well. Goofys is not a general filesystem, for example any non sequential writes would fail, including append. It's not meant to be used as generic user volume

[janibashamd]

Anyways please provide the support to create new files on NFS client. We will do testing and let you know if any issues.

[kahing]

you are free to try the dev/mknod-nfs branch which contains an incomplete and possibly not working implementation of mknod. To emphasis again NFS is much more involved and I will need to understand what kind of workloads you plan to do. Feel free to email me (email in my profile) if you prefer the discussion to be private.

[janibashamd]

I am unable to use "go get" command to use the suggested branch. Please let me know the command to use mknod branch..

The read me says goofys also works with Openstack Swift. How can you use swift back-end with EC2 credentials without S3 middleware ??. Do we need to install S3 moddleware above swift ?

[kahing]

Yes you need to use swift3. To switch to another branch, first do go get and then go to the directory where goofys is checked out, then do git checkout dev/mknod-nfs, and go install

[janibashamd]

We were able to switch to mknod branch and able to edit files on NFS server side. After NFS exports, when we try to edit existing files on NFS client side, S3 bucket is getting unmounted on server side with error "Transport endpoint is not connected".

[root@devstack dhanu]# cat dhanu.txt cat: dhanu.txt: Input/output error

[root@devstack dhanu]# touch a.txt touch: cannot touch ‘a.txt’: Input/output error

[root@vda ~]# df -kh df: ‘/mnt/goofys’: Transport endpoint is not connected df: ‘/mnt/jani’: Transport endpoint is not connected df: ‘/mnt/dhanu’: Transport endpoint is not connected

Just let me know if you need more info..

[janibashamd]

Any update would be really useful...

[gaul]

Please re-run with the flags -f --debug_fuse and include output. I will reiterate @kahing's warning that NFS support is quite involved and you should explain the workloads you want to run.

[tatobi]

It is still missing, I tested today with the latest build, creating the file: "test.txt" through an NFS mount still fails. Here is the --debug_fuse output Command on NFS client: touch /mnt/nfs/test.txt

2018/05/16 12:52:23.807638 fuse.DEBUG <-- LookUpInode 1 test.txt no such file or directory
2018/05/16 12:52:23.807664 fuse.DEBUG Op 0x00000040        connection.go:481] -> Error: "no such file or directory"
2018/05/16 12:52:23.807726 fuse.DEBUG Op 0x00000041        connection.go:396] <- MkNode (parent 1, name "test.txt")
2018/05/16 12:52:23.807769 fuse.DEBUG Op 0x00000041        connection.go:481] -> Error: "function not implemented"
2018/05/16 12:52:23.809169 fuse.DEBUG Op 0x00000042        connection.go:396] <- LookUpInode (parent 1, name "test.txt")
2018/05/16 12:52:23.810012 fuse.DEBUG Inode.LookUp 1  [test.txt]

The debug output on server: touch /mnt/s3fs/test.txt

output:

2018/05/16 13:04:54.562834 fuse.DEBUG <-- FlushFile 27 test.txt [<nil>]
2018/05/16 13:04:54.562853 fuse.DEBUG Op 0x00000099        connection.go:479] -> OK ()
2018/05/16 13:04:54.562913 fuse.DEBUG Op 0x0000009a        connection.go:396] <- SetInodeAttributes (inode 27, atime 2018-05-16 13:04:54.560288284 +0000 UTC, mtime 2018-05-16 13:04:54.560288284 +0000 UTC)
2018/05/16 13:04:54.563225 fuse.DEBUG GetAttributes 27 test.txt []
2018/05/16 13:04:54.563247 fuse.DEBUG Op 0x0000009a        connection.go:479] -> OK ()
2018/05/16 13:04:54.563283 fuse.DEBUG Op 0x0000009b        connection.go:396] <- FlushFile (inode 27)
2018/05/16 13:04:54.563556 fuse.DEBUG FlushFile 27 test.txt []
2018/05/16 13:04:54.563567 fuse.DEBUG <-- FlushFile 27 test.txt [<nil>]
2018/05/16 13:04:54.563581 fuse.DEBUG Op 0x0000009b        connection.go:479] -> OK ()
2018/05/16 13:04:54.563611 fuse.DEBUG Op 0x0000009c        connection.go:396] <- ReleaseFileHandle
2018/05/16 13:04:54.563980 fuse.DEBUG ReleaseFileHandle test.txt
2018/05/16 13:04:54.564013 fuse.DEBUG Op 0x0000009c        connection.go:479] -> OK ()
2018/05/16 13:05:01.992210 fuse.DEBUG Op 0x0000009d        connection.go:396] <- GetInodeAttributes (inode 1)
2018/05/16 13:05:01.993046 fuse.DEBUG GetAttributes 1  []
2018/05/16 13:05:01.993077 fuse.DEBUG Op 0x0000009d        connection.go:479] -> OK ()

[shvc]

It is fuse's bug, nfs client send a CREATE file request, not MkNod

Dec 26 14:47:16 vpn39 kernel: nfsd_dispatch: vers 3 proc 4
Dec 26 14:47:16 vpn39 kernel: nfsd: ACCESS(3)   8: 00010001 00000000 00000000 00000000 00000000 00000000 0x1f
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_verify(8: 00010001 00000000 00000000 00000000 00000000 00000000)
Dec 26 14:47:16 vpn39 kernel: nfsd_dispatch: vers 3 proc 3
Dec 26 14:47:16 vpn39 kernel: nfsd: LOOKUP(3)   8: 00010001 00000000 00000000 00000000 00000000 00000000 nfs1
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_verify(8: 00010001 00000000 00000000 00000000 00000000 00000000)
Dec 26 14:47:16 vpn39 kernel: nfsd: nfsd_lookup(fh 8: 00010001 00000000 00000000 00000000 00000000 00000000, nfs1)
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_compose(exp 00:25/1 /nfs1, ino=0)
Dec 26 14:47:16 vpn39 kernel: nfsd_dispatch: vers 3 proc 8
Dec 26 14:47:16 vpn39 kernel: nfsd: CREATE(3)   8: 00010001 00000000 00000000 00000000 00000000 00000000 nfs1
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_verify(8: 00010001 00000000 00000000 00000000 00000000 00000000)
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_verify(8: 00010001 00000000 00000000 00000000 00000000 00000000)
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_compose(exp 00:25/1 /nfs1, ino=0)
Dec 26 14:47:16 vpn39 kernel: nfsd_dispatch: vers 3 proc 7
Dec 26 14:47:16 vpn39 kernel: nfsd: WRITE(3)    20: 81010001 00000000 00000000 00000006 00000000 00000000 10 bytes at 0 stable
Dec 26 14:47:16 vpn39 kernel: nfsd: fh_verify(20: 81010001 00000000 00000000 00000006 00000000 00000000)
Dec 26 14:47:16 vpn39 kernel: nfsd: write complete host_err=0