search

pradeepgn / s3fs

Automatically exported from code.google.com/p/s3fs
GNU General Public License v2.0
0 stars 0 forks source link

how to access a bucket in swift from s3fs because i donot have AWSACCESSKEYID and AWSSECRETACCESSKEY #384

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
Detailed description of support request:

I have installed swift multi node configurations. I created a proxy node and 3 
storage nodes. SWIFT is working fine.
Now I thought to use s3fs.
So I installed s3 api at proxy node using link.(I didn't include s3token 
because i am not using keystone) https://github.com/fujita/swift3
Then created a new VM node (client) and installed s3fs using 
https://github.com/sagara177/s3fs-swift.

As I am emulating swift as s3, what should I use for Access Key and the Secret 
Key?
How should my s3fs command line should be?
Because swift has a concept of <token-from-x-auth-token-above> 
<url-from-x-storage-url-above>
How to connect to proxy node and mount swift containers (buckets) as a file 
system ?

Can you please help me?

===================================================================
The following information is very important in order to help us to help
you.  Omission of the following details may delay your support request or
receive no attention at all.
===================================================================
Version of s3fs being used (s3fs --version):
1.73
Version of fuse being used (pkg-config --modversion fuse):
2.8.5
System information (uname -a):
Linux localhost.localdomain 2.6.35.6-45.fc14.x86_64 #1 SMP Mon Oct 18 23:57:44 
UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

Distro (cat /etc/issue):
Fedora release 14 (Laughlin)
Kernel \r on an \m (\l)

s3fs command line used (if applicable):
s3fs myfiles /mnt/s3 -o url=https://192.168.139.130:8080/auth/v1.0 -o 
allow_other
/etc/fstab entry (if applicable):

s3fs syslog messages (grep s3fs /var/log/syslog):
Nov 10 16:12:24 localhost s3fs: init $Rev: 490 $

Original issue reported on code.google.com by naresh.b...@gmail.com on 11 Nov 2013 at 1:30

GoogleCodeExporter commented 8 years ago 
Hi,

I'm sorry for that I do not know about s3fs-swift.
I saw it's codes, it seems s3fs-swift is based s3fs v1.61.
Now, s3fs did not support swift, and I'm not prepare for it yet.

If you use s3fs(after v1.73), I think maybe you need to proxy server to swift 
proxy.
(I think this proxy should separate to access authentication and other request.)

When s3fs is run to access another server(not S3), you need to specify "url" 
option(ex. http://a.b.com).
If you specify it, when s3fs access your bucket, s3fs access 
"http://a.b.com/bucket"(not "http://bucket.a.b.com").

But you specified "https://192.168.139.130:8080/auth/v1.0", I think it was 
wrong.
Under this specified option, if you access "bucket/object", s3fs accesses to 
"https://192.168.139.130:8080/auth/v1.0/bucket/object".
Then the proxy(swift) needs to understand it.

If you need access key/secretaccesskey for S3, you can make file for it under 
/etc or $HOME(.passwd-s3fs).

If my opinion is wrong, please let me know.
(I do not understand about swift yet.)

Best Regards,

Original comment by ggta...@gmail.com on 18 Nov 2013 at 8:20

GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
Hi,
Thanks for the information.

I have installed s3fs v1.61 from below url.
git clone https://github.com/sagara177/s3fs-swift
Also created /etc/passwd-s3fs file and placed the credentials related to swift 
using enviroment variables because accesskey=system:root and secretkey=testpass

Now I get the below error, myfiles is the bucket that I have created in swift.

For 
s3fs myfiles /mnt/s3  -o url=https://192.168.139.130:8080/

Nov 19 11:46:21 localhost s3fs: curlCode: 60  msg: Peer certificate cannot be 
authenticated with known CA certificates

if I use http instead of https i.e.
s3fs myfiles /mnt/s3  -o url=http://192.168.139.130:8080/
I am able to mount (but not sure whether really mounted or not)

But on doing ls /mnt/s3
ls: reading directory /mnt/s3: Input/output error

Please help.

Thanks,
Naresh

Original comment by naresh.b...@gmail.com on 19 Nov 2013 at 8:10

GoogleCodeExporter commented 8 years ago 
Hi,

Maybe I could not support s3fs-swift, but if you use "s3fs" instead of it, I 
can support it.
For your problem, you can run s3fs with "-f" option and you will get many log 
by s3fs action.
It helps us to solve this issue.

You said s3fs could be run with http, I think it meant  s3fs was able to 
communicate to S3 once.
(Because s3fs check to access your bucket when s3fs start.)
But s3fs failed to access S3 by listing(head request) objects.
So s3fs puts logs by specifying "-f" option, we can get the reason why s3fs 
failed.

Thanks in advance for your help.

Original comment by ggta...@gmail.com on 20 Nov 2013 at 6:13

GoogleCodeExporter commented 8 years ago 
Hi,

Thanks for your inputs.

As you will be supporting for s3fs, I have installed s3fs -- V1.73 and enabled 
option -f,

Now I am getting below error, 

[root@localhost ~]# s3fs c9 /mnt/s3 -o url=https://192.168.139.130:8080/ -f
set_moutpoint_attribute(2935): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, 
mode=40755)
s3fs_check_service(2546): check services.
    CheckBucket(1927): check a bucket.
RequestPerform(1273): ### CURLE_OPERATION_TIMEDOUT
    RequestPerform(1371): ### retrying...
    RemakeHandle(1052): Retry request. [type=5][url=https://c9.192.168.139.130:8080/][path=/]
RequestPerform(1273): ### CURLE_OPERATION_TIMEDOUT
    RequestPerform(1371): ### retrying...
    RemakeHandle(1052): Retry request. [type=5][url=https://c9.192.168.139.130:8080/][path=/]
RequestPerform(1273): ### CURLE_OPERATION_TIMEDOUT
    RequestPerform(1371): ### retrying...
    RemakeHandle(1052): Retry request. [type=5][url=https://c9.192.168.139.130:8080/][path=/]
RequestPerform(1378): ### giving up
s3fs: Failed to access bucket.

Same error if I use http ..

Is the command(bucket appending before IP)e.g. https://c9.192.168.139.130:8080/ 
not understandable by swift proxy server?

I have set environment variables for AWSACCESSKEYID, and AWSSECRETACCESSKEY.
Does this credentials will be sent to s3 api? because

If I use s3curl, I am able to create a bucket and object, below are the commands

./s3curl.pl --id system:root --key testpass --put /dev/null --  -k -s -v 
https://192.168.139.130:8080/c4

./s3curl.pl --id system:root --key testpass --put s3_sample --  -k -s -v 
https://192.168.139.130:8080/c4/s3_sample

Can you please suggest

Original comment by naresh.b...@gmail.com on 21 Nov 2013 at 7:40

GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
Hi,

Thanks for the information.

Now I have installed s3fs v1.73 and exported values 
AWSACCESSKEYID = "system:root" and AWSSECRETACCESSKEY="testpass"
For
s3fs c9 /mnt/s3 -o url=http://192.168.139.130:8080/ -f
I am getting below error,
set_moutpoint_attribute(2945): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, 
mode=40755)
check_service(2546): check services.
    CheckBucket(1927): check a bucket.
RequestPerform(1278): ### CURLE_COULDNT_RESOLVE_HOST
    RequestPerform(1371): ### retrying...
    RemakeHandle(1052): Retry request. [type=5][url=http://c9.192.168.139.130:8080/][path=/]
RequestPerform(1278): ### CURLE_COULDNT_RESOLVE_HOST
    RequestPerform(1371): ### retrying...
    RemakeHandle(1052): Retry request. [type=5][url=http://c9.192.168.139.130:8080/][path=/]
RequestPerform(1278): ### CURLE_COULDNT_RESOLVE_HOST
    RequestPerform(1371): ### retrying...
    RemakeHandle(1052): Retry request. [type=5][url=http://c9.192.168.139.130:8080/][path=/]
RequestPerform(1378): ### giving up
s3fs: Failed to access bucket.

Is the above url(appending bucket before IP address 
ex.http://c9.192.168.139.130:8080/) is not understandable by swift proxy server?

Please see the below url and command that is understandable by swift3 proxy 
using s3curl

because using s3curl i am able to create buckets and objects,

./s3curl.pl --id system:root --key testpass --put /dev/null --  -k -s -v 
https://192.168.139.130:8080/c9

./s3curl.pl --id system:root --key testpass --put s3_sample --  -k -s -v 
https://192.168.139.130:8080/c9/s3_sample

CAn you please help how s3fs connects to s3 api? where it is failing?

Original comment by naresh.b...@gmail.com on 22 Nov 2013 at 9:34

GoogleCodeExporter commented 8 years ago 
Hi,

Finally after making some changes I was able to communicate to swift. I made 
changes to s3fs V1.74.

Thanks for the support.

Naresh.

Original comment by naresh.b...@gmail.com on 10 Jan 2014 at 4:10

GoogleCodeExporter commented 8 years ago 
Hi, Naresh

Thanks for replying.
I'm glad for this issue closing.

And we move s3fs project to Github(https://github.com/s3fs-fuse/s3fs-fuse), 
please see it.

Regards,

Original comment by ggta...@gmail.com on 13 Jan 2014 at 2:21