File copy error results in using all available destination disk space

[GoogleCodeExporter]

Detailed description of support request:

Copying ~530Mb of files from an S3 bucket to an EC2 Ubuntu micro instance.  It 
seems to have got stuck on one file, and that creation blew out to use up all 
available disk space ~5GB.  Subsequent files in the cp command ended up being 
zero size.

A sample ls -la result on the ubuntu ec2 ami instance drive looked like this:

-rw-r--r-- 1 myuser myuser        766 2011-02-17 07:21 p-Conglomerate P.bmp
-rw-r--r-- 1 myuser myuser        838 2011-02-17 07:20 p-Conglomerate S.bmp
-rw-r--r-- 1 myuser myuser       1126 2011-02-17 07:20 p-Diorite A.bmp
-rw-r--r-- 1 myuser myuser       1126 2011-02-17 07:21 p-Diorite.bmp
-rw-r--r-- 1 myuser myuser        982 2011-02-17 07:21 p-Dolerite A.bmp
-rw-r--r-- 1 myuser myuser        982 2011-02-17 07:20 p-Dolerite.bmp
-rw-r--r-- 1 myuser myuser        838 2011-02-17 07:20 p-Dolomite A.bmp
-rw-r--r-- 1 myuser myuser        958 2011-02-17 07:21 p-Dolomite B.bmp
-rw-r--r-- 1 myuser myuser        958 2011-02-17 07:20 p-Felsic volcanic A.bmp
-rw-r--r-- 1 myuser myuser        886 2011-02-17 07:21 p-Felsic volcanic B.bmp
-rw-r--r-- 1 myuser myuser        958 2011-02-17 07:21 p-Felsic volcanic D.bmp
-rw-r--r-- 1 myuser myuser        886 2011-02-17 07:20 p-Felsic volcanic G.bmp
-rw-r--r-- 1 myuser myuser       1126 2011-02-17 07:21 p-Felsic volcanic R.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:21 p-Fill 05.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:20 p-Fill 06.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:20 p-Fill 07.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:21 p-Fill 08.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:20 p-Fill 09.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:20 p-Fill 10.bmp
-rw-r--r-- 1 myuser myuser        438 2011-02-17 07:21 p-Fill 11.bmp
---------- 1 myuser myuser 5201956864 2011-02-21 02:17 p-Fill 12.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:20 p-Fill 13.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:21 p-Gabbro.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:20 p-Gossan.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:21 p-Granite GA.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:20 p-Granite LPK.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:21 p-Granite PK.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:20 p-Granite SKOF.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:21 p-Granite SLK.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:21 p-grid big.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:20 p-grid small.bmp
-rw-r--r-- 1 myuser myuser          0 2011-02-17 07:21 p-horizontal thick.bmp

===================================================================
The following information is very important in order to help us to help
you.  Omission of the following details may delay your support request or
receive no attention at all.
===================================================================
Version of s3fs being used (s3fs --version):
1.40

Version of fuse being used (pkg-config --modversion fuse):
2.8.4

System information (uname -a):
Linux ip-10-130-61-58 2.6.35-25-virtual #44-Ubuntu SMP Fri Jan 21 20:23:51 UTC 
2011 i686 GNU/Linux

Distro (cat /etc/issue):
Ubuntu 10.10 \n \l

s3fs command line used (if applicable):Used cloud-init script:
configs3fs:
 - &config_s3fs |
   USERNAME=myuser
   wget "http://s3fs.googlecode.com/files/s3fs-1.40.tar.gz" -P /tmp/
   cd /tmp/
   tar xvzf /tmp/s3fs-1.40.tar.gz
   cd /tmp/s3fs-1.40/
   ./configure --prefix=/usr
   make
   make install
   mkdir /mnt/s3
   chown $USERNAME:$USERNAME /mnt/s3
   chmod 777 /mnt/s3
   echo A************:0p0*********************** > /home/$USERNAME/.passwd-s3fs
   chown $USERNAME:$USERNAME /home/$USERNAME/.passwd-s3fs
   su -l $USERNAME -c "/usr/bin/s3fs my_s3_bucket /mnt/s3"
   chmod 600 /home/$USERNAME/.passwd-s3fs
   su -l $USERNAME -c "mkdir /home/$USERNAME/.wine"
   su -l $USERNAME -c "cp -a /mnt/s3/wine/drive_c /home/$USERNAME/.wine/"
   su -l $USERNAME -c "cp -a /mnt/s3/wine/*.reg /home/$USERNAME/.wine/"
   su -l $USERNAME -c "cp -a /mnt/s3/other /tmp/"

/etc/fstab entry (if applicable): NA

s3fs syslog messages (grep s3fs /var/log/syslog):

Feb 21 02:15:10 ip-10-130-61-58 s3fs: init $Rev: 312 $
Feb 21 02:16:22 ip-10-130-61-58 s3fs: ###HTTP response=500
Feb 21 02:16:26 ip-10-130-61-58 s3fs: ###retrying...
Feb 21 02:18:04 ip-10-130-61-58 s3fs: ###curlCode: 23  msg: Failed writing 
received data to disk/application
Feb 21 02:47:09 ip-10-130-61-58 s3fs: init $Rev: 312 $

Original issue reported on code.google.com by cglto...@gmail.com on 21 Feb 2011 at 3:25

[GoogleCodeExporter]

The return code 500 is one I had not seen before, S3 documentations says this:

InternalError   We encountered an internal error. Please try again. 500 Internal 
Server Error

...and I'll guess that curlCode 23 is the corresponding return code for this 
condition.

Looks like we just need to service 23 as a retry similar to several other bad 
return codes -- pretty easy fix.

Original comment by dmoore4...@gmail.com on 26 Feb 2011 at 7:04

• Changed state: Accepted

[GoogleCodeExporter]

Give r329 a try.  If it doesn't resolve the issue, please run s3fs in debug 
mode (-d option) and report back with the syslog messages. Thanks.

Original comment by dmoore4...@gmail.com on 26 Feb 2011 at 2:16

[GoogleCodeExporter]

Still waiting for a response from the original reporter. Without a response, it 
will be assumed that either this is no longer an issue or the original reporter 
no longer cares and the issue will be closed.

Original comment by dmoore4...@gmail.com on 21 Mar 2011 at 5:04

[GoogleCodeExporter]

I have run r329 and have not come across the above issue since, thank you.  The 
original problem was intermittent.
Have run the same copy over about 20 days since suggestion without any problems.

Original comment by cglto...@gmail.com on 22 Mar 2011 at 1:20

[GoogleCodeExporter]

Thank you for your response. Calling this one fixed. If the same problem pops 
up again, please feel free to re-open this issue or submit a new one.

Original comment by dmoore4...@gmail.com on 22 Mar 2011 at 1:30

• Changed state: Fixed