When s3fs is mounted on multiple machines, deletes on one machine not accurate on other machine.

[GoogleCodeExporter]

Detailed description of observed behavior:
When s3fs is mounted on multiple machines, if you delete a file from the mount 
on one machine, the mount on the other machine acts like the file is still 
there. Listing the directory correctly shows the file is gone, but trying to 
re-use the deleted file name does not work.

What steps will reproduce the problem - please be very specific and
detailed. (if the developers cannot reproduce the issue, then it is
unlikely a fix will be found)?

Create a mount on two different machines:
/mnt/s3drive

Make a directory:
mkdir test_dir

Add some files:
cp test1.txt /mnt/s3drive/test_dir
cp test2.txt /mnt/s3drive/test_dir

List directory from first machine:
ls /mnt/s3drive/test_dir
test1.txt
test2.txt

List directory from second machine:
ls /mnt/s3drive/test_dir
test1.txt
test2.txt

Everything looks good.

Delete test2.txt from first machine:
rm test2.txt

List directory from first machine:
ls /mnt/s3drive/test_dir
test1.txt

List directory from second machine:
ls /mnt/s3drive/test_dir
test1.txt

Everything looks good.

Now, on second machine, try to re-use the name of the file that was deleted:
cp test1.txt test2.txt
cp: overwrite ‘test2.txt’? y
cp: cannot fstat ‘test2.txt’: No such file or directory
cp: failed to close ‘test2.txt’: No such file or directory

It acts like test2.txt still exists and asks if it should be overwritten and 
then it breaks.

===================================================================
The following information is very important in order to help us to help
you.  Omission of the following details may delay your support request or
receive no attention at all.
===================================================================
Version of s3fs being used (s3fs --version):
1.74

Version of fuse being used (pkg-config --modversion fuse):

System information (uname -a):
Linux 3.11.7-200.fc19.x86_64 #1 SMP Mon Nov 4 14:09:03 UTC 2013 x86_64 x86_64 
x86_64 GNU/Linux

Distro (cat /etc/issue):
Fedora release 19 (Schrödinger’s Cat)

s3fs command line used (if applicable):

/etc/fstab entry (if applicable):

s3fs syslog messages (grep s3fs /var/log/syslog):

Original issue reported on code.google.com by penguin...@gmail.com on 16 Dec 2013 at 5:31