folder creation fails while rsyncing

[GoogleCodeExporter]

While rsyncing a tree with many folders, some folders don't get properly 
created, i.e. in the aws console i see only a file with the folder name, but no 
folder. 
When a file inside that folder gets created, i see in the log:
Feb  3 15:59:08 e6510 s3fs: HTTP response code 404
Feb  3 15:59:08 e6510 s3fs: HTTP response code 404 was returned
Feb  3 15:59:08 e6510 s3fs: Now returning ENOENT

I'm running s3fs-1.35 on Ubuntu 10.10. I tried with both US and Ireland based 
buckets.

Original issue reported on code.google.com by matteo.b...@gmail.com on 3 Feb 2011 at 4:10

[GoogleCodeExporter]

When accessing the bucket through a s3fs mount, do you see the directories as 
expected?

If so, then please see comment 1 in issue #73

Original comment by dmoore4...@gmail.com on 8 Feb 2011 at 4:47

[GoogleCodeExporter]

No, the directories are not shown. From my understanding, here's what happens:
1. s3fs creates a "placeholder" empty file for every empty directory
2. sometimes this creation fails, thus:
3. when uploading a file to a certain directory, s3fs checks if the folder 
(placeholder file) exists
4. the folder does not exist, hence the upload failure.
Does this makes sense?

Original comment by matteo.b...@gmail.com on 8 Feb 2011 at 2:00

[GoogleCodeExporter]

Eventual Consistency?

Have you seen this problem with a US-WEST bucket?

Do subsequent rsync commands eventually end in success?

Which options are you using to rsync?

Hopefully by reading the above referenced issue that you understand that 
directories are represented by zero length files -- this is s3fs's convention 
for a "file system" which does not have a native concept of directories.  The 
directory created is not a placeholder, but the actual directory 
representation.  

Through observation, I have noticed that rsync (not s3fs) first creates the 
directory skeleton on the receiving side of things -- the actual directories, 
not placeholders.

Your symptoms do point to E.C. rsync is looking for something that it created 
but it isn't there yet.  Run rsync with several -v options to see exactly what 
it is failing upon.  If it's the missing directory that supposedly was all 
ready created, check back in a few minutes to see if it showed up.

Original comment by dmoore4...@gmail.com on 8 Feb 2011 at 3:08

[GoogleCodeExporter]

No response from the reporter in a month. It is assumed that this is no longer 
an issue, or its expected behavior (rsync or EC), or the reporter doesn't care 
any more. We can't effectively help resolve issues without enough detail. We 
need to detail to attempt to reproduce the potential issue. Closing this one.

Original comment by dmoore4...@gmail.com on 9 Mar 2011 at 5:12

• Changed state: WontFix