kahing / goofys

a high-performance, POSIX-ish Amazon S3 file system written in Go
Apache License 2.0
5.2k stars 522 forks source link

Starting in foreground, exits after a file is copied? #270

Open jamshid opened 6 years ago

jamshid commented 6 years ago

I installed goofys version 0.19.0-943e017724ea820eb4185419ef3c41d6f921a324 on my macOS 10.13.3 but it seems to exit immediately after using cp to copy a file into my S3-compatible virtual directory.

$ cd /tmp/drop/foo
$ cp ~/Movies/bar.mp4 .
cp: .../bar.mp4: could not copy extended attributes to ./bar.mp4: Permission denied
cp: ./bar.mp4: Socket is not connected

I had started goofys with:

$ goofys -f --debug_fuse --debug_s3 --endpoint https://mystorage.example.com drop /tmp/drop
...
2018/02/01 15:18:46.829225 s3.DEBUG {
  Bucket: "drop",
  ETag: "0a82f78105cc86f6dbd2915b39ffdba3-125",
  Key: "foo/bar.mp4",
  Location: ""
}
2018/02/01 15:18:46.829241 fuse.DEBUG <-- FlushFile 492 foo/bar.mp4 [<nil>]
2018/02/01 15:18:46.829271 fuse.DEBUG Op 0x00000007        connection.go:479] -> OK ()
2018/02/01 15:18:46.829315 main.INFO Successfully exiting.

Why does it exit? When I check the S3-compatible server the file is there.

Even worse, my system is left in a bad state and I can't restart goofys.

$ ls /tmp/drop
ls: drop: Device not configured
$ goofys -f --debug_fuse --debug_s3 --endpoint https://mystorage.example.com drop /tmp/drop
...
2018/02/01 15:29:10.413894 main.FATAL Mounting file system: Mount: newConnection: Init: Reading init op: EOF

Is there a process I can kill or extension I can unload to avoid rebooting?

jamshid commented 6 years ago

Ah if I umount then I can run goofys again without rebooting.

$ umount /tmp/drop

Did a little more testing, goofys exits when I copy a large file (500MB) but doesn't seem to exit after copying a small file (50MB).

kahing commented 6 years ago

this is really strange, could you include more of the log?

jamshid commented 6 years ago

Attaching a scrubbed log file of the successful multipart upload followed by unexpected successful exit, running on latest macos installed with brew.

goofys.log The file was uploaded successfully, I see it if I rerun goofys and ls -ltra:

-rw-r--r--@  1 user  staff  655141034 Mar 11 14:29 RUDOLF_THE_RED_NOSED_REINDEER.mp4
jrslv commented 6 years ago

I have the same issue when using goofys (0.19.0) with osxfuse (3.8.2) to manage my photos and videos in S3. Uploading photos works OK, but videos (100M+) usually fail, meaning goofys just exits. In my case it does not matter if I run in in the front -f or normally.

This is end of the log:

2018/07/25 12:54:10.973233 fuse.DEBUG <-- FlushFile 32 2017/July/IMG_4372.MOV [<nil>]
2018/07/25 12:54:10.973335 fuse.DEBUG Op 0x00000010        connection.go:479] -> OK ()
2018/07/25 12:54:10.973445 main.INFO Successfully exiting.

My workflow nowadays consists of a lot of goofys restarts :-(