Pseudo directory not recognized by horizon

supnobita commented 8 years ago

Context

svfs version : svfs_0.5.2
storage provider : Swift - Openstack (pcs)
product : svfs

the first issue is: file size with du command is always zero (but ls -la is have file size)

Steps to reproduce this issue :

mount swift on your machine
cd to one container
type: du -sh *
Results you expected :

/home/ftpuser/cloudfuse/123# du -sh * 12K 10.jpg 476M thumuc

Results you observed :

/home/ftpuser/cloudfuse/123# du -sh * 0 10.jpg 0 thumuc

Debug log :

Additional information :

and the second issue is: pseudo directory, may be you not implement function create pseudo directory well.

Steps to reproduce this issue :

mount swift on your machine
cd to one container (ex: container1)
mkdir one directory on this container (ex: dir1)
use swift client type: swift stat comtainer1 dir1

Results you expected

Account: AUTH_751c471132a54a3b9d1730baf6a52cb3 Container: container1 Object: dir1/ Content Type: httpd/unix-directory Content Length: 0 Last Modified: Wed, 08 Jun 2016 01:40:34 GMT ETag: d41d8cd98f00b204e9800998ecf8427e Accept-Ranges: bytes X-Timestamp: 1465350033.69469 X-Trans-Id: txf2c6546b946c41f6a12c7-005757e4b8

Results you observed

Account: AUTH_751c471132a54a3b9d1730baf6a52cb3 Container: container1 Object: dir1 Content Type: application/directory Content Length: 0 Last Modified: Wed, 08 Jun 2016 07:26:52 GMT ETag: d41d8cd98f00b204e9800998ecf8427e Accept-Ranges: bytes X-Timestamp: 1465370811.72025 X-Trans-Id: tx76b30e75ae41455d841a8-005757e4d8

Please read the difference at: Content Type: That make the swift think the dir1 is a object not a directory (when we get it via API).

xlucas commented 8 years ago

Hello,

Please stick with reporting one issue per ticket, this template has been added for this purpose.

First issue : svfs handles stat() and statfs() since version 0.6.1, check the release section.

Second issue : content-type application/directory is a defacto standard used by many swift clients. Not honoring this would break them. So this definitely should come as a mount option if implemented.

supnobita commented 8 years ago

Hi lucas, I upgraded svfs 7.0 and this fix first issue. And the second issue: I think have a difference in contten-type between swift client and Horizon. When create pseudo directory dir1 on SVFS, Horizon see one file name dir1 (size 0 Byte, content-type: application/directory), When create dir1 on Horizon, it have content-type: httpd/unix-directory, that inconsistent make application confuse. Could you suggest me how can I fix that ? May be change content-type: httpd/unix-directory ? And this fix may have any issue ?

Thanks

supnobita commented 8 years ago

Could you give me some instruction to complie your code ?

xlucas commented 8 years ago

Actually that observation doesn't come from the fact that content-type are different.

What happens is :

svfs create a 0 byte object with content-type application/directory
when listing container's content, a list of subdirectories is generated but the application/directory object is not interpreted as a pseudo-directory as it should

I'll double-check that.

xlucas commented 8 years ago

@supnobita Are you familiar with golang projects ?

You just need to install golang, then go get github.com/ovh/svfs. After a change in the code base, run go install github.com/ovh/svfs.

The svfs binary will be placed at $GOPATH/bin/svfs.

supnobita commented 8 years ago

I think when create a pseudo directory, svfs need a delimiter "/" like http://loose-bits.com/2010/12/rackspace-cloud-files-and-pseudo.html I testing in my server create a object with delimiter "/" : curl -i http://controller:8080/v1/AUTH_751c471132a54a3b9d1730baf6a52cb3/minhuc/minhuc-createdir/ -X PUT -H "Content-Length: 0" -H "Content-Type: application/directory; " -H "X-Auth-Token: xxxx" And it work fine ! Svfs can work normal on pseudo dir I created. But I'm not familiar with golang. I get your project and type go install but it give errror, it can find sync.pool, so i update golan 1.4, but it have many error like: bazil.org/fuse/buffer.go:3:8: cannot find package "unsafe" in any of: /root/work/pkg/src/unsafe (from $GOROOT) /root/work/src/unsafe (from $GOPATH) bazil.org/fuse/error_linux.go:4:2: cannot find package "syscall" in any of: /root/work/pkg/src/syscall (from $GOROOT) /root/work/src/syscall (from $GOPATH) bazil.org/fuse/fuse.go:112:2: cannot find package "time" in any of: /root/work/pkg/src/time (from $GOROOT) /root/work/src/time (from $GOPATH) What i should in your code base ? Could you give me some advice ? Many thanks

xlucas commented 8 years ago

@supnobita Sounds like you have not set GOROOT. You'll also need go 1.5+ (1.6 recommanded) to use vendored dependencies.

supnobita commented 8 years ago

@xlucas I complied and installed svfs. I change https://github.com/supnobita/svfs/blob/master/svfs/directory.go#L255 to:

if err := SwiftConnection.ObjectPutBytes(d.c.Name, absPath+**"/"**, nil, dirContentType); err != nil {

It make Horizon understand my new dir, but I have trouble with delete folder. I use SVFS + pureftp to help user upload file to Swift via FTP. And when I delete folder, it only delete a folder object (object with size 0) but not delete object in dir (ex: abc/object.txt, it delete abc/, not delte object.xt). I found that, the rmdir command in FTP put the task delete all child object of directory to file system. But Fuse or Svfs not implement. When I use rm -rf abc/ it work well. And may be we have a bug with Rename, When I delete folder it bun out error like that http://pastebin.com/WjxcLhWh I don't understand why i remove folder have a rename error ?

xlucas commented 8 years ago

Can you checkout branch pseudo-directory and check that it fixes your issue ?

ovh / svfs