Closed vgadhadaran closed 7 years ago
Hello,
please provide more information, how do you start riofs
, which command arguments do you use ?
Try to run riofs
with -l [log_file_name]
command option to save logs, then paste it here.
riofs ritestdemo1 /tmp/wow -c /usr/local/etc/riofs/riofs.conf.xml -o "nomultipart,sigv2,allow_other,nonempty" -v --log-file=/var/log/riofs.log
18:01:35 [dir_tree](dir_tree_file_write dir_tree.c:1499) [ino: 16, fop: 0xf4bcb0] write inode, size: 4096, off: 524279808 18:01:35 [fio](fileio_write_buffer file_io_ops.c:680) [ino: 16] Write buf size: 524283904 18:01:35 [cmng](cache_mng_store_file_buf cache_mng.c:419) [ino: 16] Written [524279808:4096] bytes, result: OK 18:01:35 [fuse](rfuse_write_cb rfuse.c:717) [req: 0xf53470] write_cb success: YES 18:01:35 [dir_tree](dir_tree_on_buffer_written_cb dir_tree.c:1445) [ino: 16, fop: 0xf4bcb0] Buffer written, count: 4096 18:01:35 [fuse](rfuse_write rfuse.c:732) [ino: 16, fi: 0x7fffca8f0ee0] write inode, size: 4096, off: 524283904 18:01:35 [dir_tree](dir_tree_file_write dir_tree.c:1499) [ino: 16, fop: 0xf4bcb0] write inode, size: 4096, off: 524283904 18:01:35 [fio](fileio_write_buffer file_io_ops.c:680) [ino: 16] Write buf size: 524288000 18:01:35 [cmng](cache_mng_store_file_buf cache_mng.c:419) [ino: 16] Written [524283904:4096] bytes, result: OK 18:01:35 [con](http_connection_acquire http_connection.c:174) [con: 0xf4c150] Connection object is acquired! 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (19msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 1 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (11msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 2 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 3 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (1msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 4 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 5 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (1msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 6 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 7 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 8 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 9 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 10 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 11 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 12 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 13 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 14 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 15 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 16 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 17 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 18 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 19 of 20 18:01:35 [con](http_connection_make_request http_connection.c:814) [con: 0xf4c150] POST /ritestdemo1/500m1?uploads bucket: ritestdemo1, host: cloud-poc.hcpdevn1.xxx.xxx, out_len: 0 18:01:35 [con](http_connection_on_responce_cb http_connection.c:464) [con: 0xf4c150] Got HTTP response from server! (2msec) 18:01:35 [con](http_connection_on_responce_cb http_connection.c:591) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented). AWS message: The request requires functionality that is not implemented in the current release 18:01:35 [con](http_connection_on_responce_cb http_connection.c:602) [con: 0xf4c150] Server returned HTTP error: 501 (Not Implemented)! Retry ID: 20 of 20 18:01:35 [con](http_connection_on_responce_cb http_connection.c:605) [con: 0xf4c150] Reached the maximum number of retries ! 18:01:35 [con](http_connection_release http_connection.c:183) [con: 0xf4c150] Connection object is released! 18:01:35 [fio](fileio_write_on_multipart_init_cb file_io_ops.c:598) [ino: 16, con: 0xf4c150] Failed to get multipart init data from the server ! 18:01:35 [fuse](rfuse_write_cb rfuse.c:717) [req: 0xf52a90] write_cb success: NO 18:01:35 [dir_tree](dir_tree_on_buffer_written_cb dir_tree.c:1445) [ino: 16, fop: 0xf4bcb0] Buffer written, count: 0 18:01:35 [fuse](rfuse_getxattr rfuse.c:927) getxattr for: 16 attr name: security.capability size: 0 18:01:35 [dir_tree](dir_tree_getxattr dir_tree.c:2261) [ino: 16] Getting Xattributes .. 18:01:35 [dir_tree](dir_tree_getxattr dir_tree.c:2285) Xattr :security.capability not supported! 18:01:35 [fuse](rfuse_getxattr_cb rfuse.c:905) [ino: 16] getattr_cb success: NO str: (null) 18:01:35 [fuse](rfuse_write rfuse.c:732) [ino: 16, fi: 0x7fffca8f0ee0] write inode, size: 4096, off: 524283904 18:01:35 [dir_tree](dir_tree_file_write dir_tree.c:1499) [ino: 16, fop: 0xf4bcb0] write inode, size: 4096, off: 524283904 18:01:35 [fio](fileio_write_buffer file_io_ops.c:671) [ino: 16] Write call with offset 524283904 is not allowed ! 18:01:35 [fuse](rfuse_write_cb rfuse.c:717) [req: 0xf4fe60] write_cb success: NO 18:01:35 [dir_tree](dir_tree_on_buffer_written_cb dir_tree.c:1445) [ino: 16, fop: 0xf4bcb0] Buffer written, count: 0 18:01:35 [fuse](rfuse_flush rfuse.c:1047) [0xf4e7d0][req: 0xf52a90] flush ino: 16 18:01:35 [fuse](rfuse_flush_cb rfuse.c:1033) [req: 0xf52a90] flush_cb success: YES 18:01:35 [fuse](rfuse_release rfuse.c:676) [ino: 16, fi: 0x7fffca8f0ee0] release inode, flags: 32769 18:01:35 [dir_tree](dir_tree_file_release dir_tree.c:1361) [ino: 16, fop: 0xf4bcb0] dir_tree_file_release 18:01:35 [fio](fileio_release_on_part_con_cb file_io_ops.c:321) [ino: 16, con: 0xf4c150] Releasing fop. Size: 524288000 18:01:38 [fio](fileio_release_on_part_con_cb file_io_ops.c:341) [ino: 16, con: 0xf4c150] UploadID is not set, aborting operation !
Iam writing data to the file on non-RioFS folder, then copy file to RioFS mounted folder.
As we have massive amount of data in terrabytes we need move to s3 storage. All our data varies from 250mb, 500mb, 1 gb/5/10/100/250 gb is sizes. We had success only moving 250 mb of data although it took 2 hours. The above failure happened when moving 500mb of data. any help appreciated.
currently i see write inode size is 4096 is there any way to increase it as it takes hugh time to copy massive data?
Most likely you are trying to use RioFS and AWS region with only version 4 support (please check http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region), unfortunately at this point RioFS supports only version 2 regions.
I am connecting to Hitachi content platform and not to AWS.
I'm afraid I can't help you with this issue, as I never tested RioFS against "Hitachi content platform". As you can see in the log file - server returns many "501" HTTP status codes, most likely Hitachi uses modified or custom S3 REST API.
I installed the riofs on rhel server by following the below installation steps
sudo yum groupinstall "Development Tools" sudo yum install glib2-devel fuse-devel libevent-devel libxml2-devel openssl-devel libmagic-dev cp /tmp/libevent-2.0.22-stable.tar.gz /usr/local/src cd /usr/local/src tar -zxvf libevent-2.0.22-stable.tar.gz cd libevent-2.0.22-stable
./configure make sudo make install sudo echo "/usr/local/lib/" > /etc/ld.so.conf.d/riofs.conf sudo ldconfig export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig cd cd /tmp cp -r riofs-master /usr/local/src/ cd /usr/local/src cd riofs-master/ chmod +x autogen.sh ./autogen.sh ./configure make sudo make install
Now when I try to move the data, it throws below error,
[root@k8002166 who]# mv 500m1 /tmp/wow mv: writing `/tmp/wow/500m1': No such file or directory
Anythoughts?