skoobe / riofs

Userspace S3 filesystem
GNU General Public License v3.0
393 stars 60 forks source link

riofs and google cloud storage s3 compatibility #150

Open EssentialMix opened 7 years ago

EssentialMix commented 7 years ago

Hi i am trying to mount GCS using riofs with Interoperability API enabled for bucket, i was able to mount bucket using http, but dirlist of empty. Anybody able to use riofs with gcs? 23:11:28 [main] (main main.c:753) Using config file: /usr/local/etc/riofs/riofs.conf.xml 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f30b0] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_make_request http_connection.c:814) [con: 0x17f30b0] GET /my-bucket/?acl bucket: my-bucket, host: storage.googleapis.com, out_len: 0 23:11:28 [con] (http_connection_on_response_cb http_connection.c:464) [con: 0x17f30b0] Got HTTP response from server! (87msec) 23:11:28 [con] (http_connection_release http_connection.c:183) [con: 0x17f30b0] Connection object is released! 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f43d0] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f4980] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f4f80] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f5550] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f5b50] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f6120] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f66f0] Connecting to storage.googleapis.com:80 23:11:28 [con] (http_connection_init http_connection.c:79) [con: 0x17f6cc0] Connecting to storage.googleapis.com:80 23:11:28 [cmng] (cache_mng_create cache_mng.c:84) Maximum cache size (bytes): 1073741824 23:11:28 [dir_tree] (dir_tree_add_entry dir_tree.c:256) [ino: 1] Creating new DirEntry: /, fullpath: , mode: 16877 time: 2017-10-12 23:11:28 23:11:28 [dir_tree] (dir_tree_create dir_tree.c:147) DirTree created 23:11:35 [fuse] (rfuse_getattr rfuse.c:515) [ino: 1] getattr 23:11:35 [dir_tree] (dir_tree_getattr dir_tree.c:1229) [ino: 1] Getting attributes.. 23:11:35 [fuse] (rfuse_getattr_cb rfuse.c:488) [ino: 1] getattr_cb, success: YES 23:11:36 [fuse] (rfuse_getattr rfuse.c:515) [ino: 1] getattr 23:11:36 [dir_tree] (dir_tree_getattr dir_tree.c:1229) [ino: 1] Getting attributes.. 23:11:36 [fuse] (rfuse_getattr_cb rfuse.c:488) [ino: 1] getattr_cb, success: YES 23:11:37 [fuse] (rfuse_opendir rfuse.c:398) [ino: 1] opendir 23:11:37 [fuse] (rfuse_readdir rfuse.c:472) [ino: 1] readdir inode, size: 4096, off: 0 23:11:37 [dir_tree] (dir_tree_fill_dir_buf dir_tree.c:631) [ino: 1] Requesting directory buffer: [4096: 0] 23:11:37 [dir_tree] (dir_tree_fill_dir_buf dir_tree.c:710) [ino: 1] Directory cache is expired, getting a fresh list from the server ! 23:11:37 [dir_tree] (dir_tree_start_update dir_tree.c:307) UPDATED CURRENT AGE: 1 23:11:37 [con_dir] (http_connection_get_directory_listing http_connection_dir_list.c:352) [ino: 25123664, con: 0x17f5b50] Getting directory listing for: >><< 23:11:37 [con] (http_connection_acquire http_connection.c:174) [con: 0x17f5b50] Connection object is acquired! 23:11:37 [con] (http_connection_make_request http_connection.c:814) [con: 0x17f5b50] GET /my-bucket/?delimiter=/&max-keys=10000&prefix= bucket: my-bucket, host: storage.googleapis.com, out_len: 0 23:11:38 [con] (http_connection_on_response_cb http_connection.c:464) [con: 0x17f5b50] Got HTTP response from server! (755msec) 23:11:38 [con_dir] (http_connection_on_directory_listing_data http_connection_dir_list.c:319) [ino: 1, con: 0x17f5b50] Directory listing done ! 23:11:38 [dir_tree] (dir_tree_fill_on_dir_buf_cb dir_tree.c:472) [ino: 1 req: 0x17f1cd0] Dir fill callback: SUCCESS 23:11:38 [fuse] (rfuse_add_dirbuf rfuse.c:433) [ino: 1] add_dirbuf, name: . 23:11:38 [fuse] (rfuse_add_dirbuf rfuse.c:433) [ino: 1] add_dirbuf, name: .. 23:11:38 [dir_tree] (dir_tree_fill_on_dir_buf_cb dir_tree.c:503) [ino: 1] Total entries in directory: 0 23:11:38 [fuse] (rfuse_readdir_cb rfuse.c:453) readdir_cb success: YES, buf_size: 64, size: 4096, off: 0 23:11:38 [dir_tree] (dir_tree_fill_on_dir_buf_cb dir_tree.c:551) [ino: 1] Dir cache updated: 1507849898, items: 0 23:11:38 [dir_tree] (dir_tree_stop_update dir_tree.c:364) [ino: 1] Removing old DirEntries for: .. 23:11:38 [con] (http_connection_release http_connection.c:183) [con: 0x17f5b50] Connection object is released!

wizzard commented 7 years ago

RioFS is only compatible with AWS S3, it does not work with any other Cloud providers.

EssentialMix commented 7 years ago

Yes, but GCS support S3 API. I am curious if there is a way to make it work. But again thanks for your answer. If i will figure out, i will update this issue.

wizzard commented 6 years ago

I'm quite sure it won't work without modifying RioFS code.