kahing / goofys

a high-performance, POSIX-ish Amazon S3 file system written in Go
Apache License 2.0
5.2k stars 523 forks source link

goofys panic when ls with more than 5000 files under the bucket #335

Closed jimmyyan closed 6 years ago

jimmyyan commented 6 years ago

Hi, kahing I'm trying to use goofys to mount cos(tencent object storage) . My mount command is as follows: ./goofys --endpoint http://cos.ap-beijing-1.myqcloud.com --debug_s3 --debug_fuse --dir-mode 0777 --file-mode 0777 -f --stat-cache-ttl 10s --type-cache-ttl 10s cgilogtj-1253960454 /mnt/goofyscgilogtj/

goofys will panic accidentally,when execute ls after cache expired. here is the stack below:

2018/07/03 15:02:51.912017 fuse.DEBUG find inode 1 [4181 4863] 2018/07/03 15:02:51.912042 fuse.DEBUG Ref 14933 4863 [0] 2018/07/03 15:02:51.912092 fuse.DEBUG <-- LookUpInode 1 4863 2018/07/03 15:02:51.912117 fuse.DEBUG Op 0x00008816 connection.go:478] -> OK (inode 14933) 2018/07/03 15:02:51.912139 fuse.DEBUG Op 0x00008817 connection.go:478] -> OK () 2018/07/03 15:02:51.912161 fuse.DEBUG Op 0x00008818 connection.go:395] <- ForgetInode (inode 4339) 2018/07/03 15:02:51.912172 fuse.DEBUG DeRef 4339 4863 [1 1] 2018/07/03 15:02:51.912188 fuse.DEBUG Op 0x00008818 connection.go:478] -> OK () 2018/07/03 15:02:51.912209 fuse.DEBUG Op 0x00008819 connection.go:395] <- GetXattr (inode 14933, name "security.capability", name security.capability) 2018/07/03 15:02:51.912254 s3.DEBUG DEBUG: Response s3/HeadObject Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 404 Not Found Connection: keep-alive Content-Type: application/xml Date: Tue, 03 Jul 2018 07:02:51 GMT Server: tencent-cos X-Cos-Request-Id: NWIzYjFmOWJfNTViMjM1MGFfM2I1MF81OWU4NDc= X-Cos-Trace-Id: OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODc0OWRkZjk0ZDM1NmI1M2E2MTRlY2MzZDhmNmI5MWI1OTBjYzE2MjAxN2M1MzJiOTdkZjMxMDVlYTZjN2FiMmI0YTgzZGM0ZWRlMWE0OGFjYmNkMDQ0M2Q4ZTUwYTdiMjg= Content-Length: 0


2018/07/03 15:02:51.912280 s3.DEBUG DEBUG: Validate Response s3/HeadObject failed, not retrying, error NotFound: Not Found status code: 404, request id: , host id: panic: Unknown inode: 14933

goroutine 88958 [running]: github.com/kahing/goofys/internal.(Goofys).getInodeOrDie(0xc42018bad0, 0x3a55, 0xc4212f94a0) /root/jimmyyan/src/src/github.com/kahing/goofys/internal/goofys.go:421 +0x126 github.com/kahing/goofys/internal.(Goofys).GetXattr(0xc42018bad0, 0xa00a60, 0xc420b264e0, 0xc421bc1140, 0x0, 0x0) /root/jimmyyan/src/src/github.com/kahing/goofys/internal/goofys.go:465 +0x5b github.com/jacobsa/fuse/fuseutil.(fileSystemServer).handleOp(0xc420121c20, 0xc4202c85b0, 0xa00a60, 0xc420b264e0, 0x896e80, 0xc421bc1140) /root/jimmyyan/src/src/github.com/jacobsa/fuse/fuseutil/file_system.go:211 +0x4ad created by github.com/jacobsa/fuse/fuseutil.(fileSystemServer).ServeOps /root/jimmyyan/src/src/github.com/jacobsa/fuse/fuseutil/file_system.go:121 +0x176

It seems that it find the inode which has been forgotten by kernel.

Could you figure out what the problem is?

kahing commented 6 years ago

probably related to #306 (the issue's summary is wrong). I've reverted the broken commit so you can try the latest master

jimmyyan commented 6 years ago

Thanks for replying, I'll try the latest master