search

kahing / goofys

a high-performance, POSIX-ish Amazon S3 file system written in Go
Apache License 2.0
5.19k stars 521 forks source link

Issue with cp file with specific file format #424

Open ganeshmailbox opened 5 years ago

ganeshmailbox commented 5 years ago

We are having issues in copying some specific file formats to mounted S3 bucket. Please note that we are able to copy CSV, txt, log files etc without any issues and also able to do SCP and dump file in here.

But when we are trying to copy (using cp command) SAS datasets (.sas7bdat files). It’s failing with error as below. cp: error writing ‘/s3-mybucket/data/sourcedata/output/mysasdata.sas7bdat’: Operation not supported cp: failed to close ‘/s3-mybucket/data/sourcedata/output/mysasdata.sas7bdat’: Operation not supported

Here are the setup details It is running on CentOS.

[z@ Output_old]$ rpm -qa | grep fuse fuse-2.9.2-11.el7.x86_64 fuse-libs-2.9.2-11.el7.x86_64 fuseiso-20070708-15.el7.x86_64 gvfs-fuse-1.36.2-1.el7.x86_64 fuse-devel-2.9.2-11.el7.x86_64 glusterfs-fuse-3.12.2-18.el7.x86_64 [zOutput_old]$ rpm -qa | grep curl python-pycurl-7.19.0-17.el7.x86_64 libcurl-7.29.0-51.el7.x86_64 libcurl-devel-7.29.0-51.el7.x86_64 curl-7.29.0-51.el7.x86_64

And here is the command that I used to mount the bucket. /sasmount/goofys --dir-mode=0777 --file-mode=0666 -o allow_other mybucket /s3-mybucket

Could you please suggest what might be going wrong here.

kahing commented 5 years ago

is that a sparse file by any chance?

ezman commented 3 years ago

I am also seeing this issue. Text files and small binary copy ok. But large database backup files, fail with the same error message the original poster sees.

Are sparse data files not supported?

I can do a full debug trace if need be. My syslog has the following messages :

Nov 13 15:33:36 myserver /usr/bin/goofys[9514]: fuse.ERROR WriteFile: only sequential writes supported 11 GLDbackup/BLAHbLahBLAH.bkp [4096 8192] Nov 13 15:33:36 ftp-uat /usr/bin/goofys[9514]: fuse.ERROR *fuseops.WriteFileOp error: operation not supported Nov 13 15:33:36 ftp-uat /usr/bin/goofys[9514]: fuse.ERROR *fuseops.FlushFileOp error: operation not supported

dotslash commented 3 years ago

Can you try cp --sparse=never a b (On os-x gcp --sparse=never a b)

dotslash commented 3 years ago

Yes, sparse files are not supported

lnicola commented 2 years ago

I also ran into this with one small difference: I don't get any ENOTSUPP error, just corrupted files.