ghost folders

[GoogleCodeExporter]

I try to use s3fs with a drupal website, a classical LAMP application, on 
Ubuntu LTS. s3fs is version r177.
Drupal is not aware of s3 stuff, files are used as classical UNIX files.

First step, drupal create a new file. Dumb folder was created :
(dump is done with s3sync)
--------------------
imagefield_thumbs
imagefield_thumbs/img
imagefield_thumbs/img/sessions
imagefield_thumbs/img/sessions/wall1-1920x1200.jpg
img
img/sessions
img/sessions/wall1-1920x1200.jpg

Drupal does something else, and dumb folders vanishes :
--------------------
imagefield_thumbs/img/sessions/wall1-1920x1200.jpg
img/sessions/wall1-1920x1200.jpg

"ls" in s3 mount path show nothing, logically, there is no first level dummy 
folder. If do a "mkdir -p" to add missing folder, files inside this path is 
coming back. I can't figure how drupal can remove dummy folder without 
erasing files inside, I guess s3fs does some cleanings.

Original issue reported on code.google.com by mathieu....@gmail.com on 12 May 2009 at 3:36

[GoogleCodeExporter]

s3fs_readdir show the ghost folder, but s3fs_getattr see nothing while S3 
return him 
404 (wich is true, it's ghost folder, it doesn't exist, but have sons). I'm 
working on a 
patch : if a folder doesn't exist but is listed as a son of its father, then, 
it's a dummy 
folder.

Original comment by mathieu....@gmail.com on 15 May 2009 at 3:41

[GoogleCodeExporter]

s3fs r177 doesn't handle well rmdir. It doesn't check if the folder is empty.
Here is the patch :

22a23
> 
985c986,1013
<       cout << "unlink[path=" << path << "]" << endl;
---
>       cout << "rmdir[path=" << path << "]" << endl;
> 
>       {
>               string resource = urlEncode ("/" + bucket);
>               string query = "delimiter=/&prefix=";
> 
>               if (strcmp(path, "/") != 0)
>                       query += urlEncode(string(path).substr(1) + "/");
> 
>               query += "&max-keys=50";
> 
>               string url = host + resource + "?"+ query;
> 
>               string responseText;
> 
>               auto_curl curl;
>               curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
>               curl_easy_setopt(curl, CURLOPT_FAILONERROR, true);
>               curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, true);
>               curl_easy_setopt(curl, CURLOPT_WRITEDATA, &responseText);
>               curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeCallback);
> 
>               auto_curl_slist headers;
>               string date = get_date();
>               headers.append("Date: "+date);
>               headers.append("Authorization: AWS
"+AWSAccessKeyId+":"+calc_signature("GET", "", date, headers.get(), resource));
> 
>               curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers.get());
986a1015,1028
>               VERIFY(my_curl_easy_perform(curl.get()));
>               cout << endl << responseText << endl;
>               if (responseText.find ("<CommonPrefixes>") != std::string::npos 
||
responseText.find ("<ETag>") != std::string::npos )
>               {
>                       // directory is not empty
>                       cout << "[path=" << path << "] not empty" << endl;
>                       return -ENOTEMPTY;
>               }
>               else
>               {
>                       // directory is empty
>               }
>       }
> 

Original comment by mathieu....@gmail.com on 27 May 2009 at 8:49

[GoogleCodeExporter]

Incorporating above patch.  In the future, please use "diff -u" instead of 
plain diff.  It gives the patch utility a better shot at applying that patch. 
Thanks!

Original comment by dmoore4...@gmail.com on 18 Oct 2010 at 11:06

• Changed state: Started

[GoogleCodeExporter]

Incorporated patch and tested, rmdir fails as it should upon attempting to 
rmdir a non-empty directory:

% ls -l newdir
total 1
-rw-r--r-- 1 root root 0 Oct 18 19:42 newfile

% rmdir newdir
rmdir: failed to remove `newdir': Directory not empty

committed in revision 201

Original comment by dmoore4...@gmail.com on 19 Oct 2010 at 1:45

• Changed state: Fixed