search

gluster / gluster-kubernetes

GlusterFS Native Storage Service for Kubernetes
Apache License 2.0
875 stars 389 forks source link

s3 auth failed. #561

Closed chenyg0911 closed 5 years ago

chenyg0911 commented 5 years ago

use gk-deploy deploy the gluster-s3 on native k8s. Can't pass the test. all pod runing.

kubectl get pod
NAME                                     READY   STATUS    RESTARTS   AGE
gluster-s3-deployment-74cf744cd6-n4tcx   1/1     Running   0          18h
glusterfs-254w6                          1/1     Running   0          18h
glusterfs-lhzn6                          1/1     Running   0          18h
glusterfs-r8w4d                          1/1     Running   0          18h
heketi-7495cdc5fd-44t58                  1/1     Running   0          18h

the pod gluster-s3-depolyment env:

kubectl exec gluster-s3-deployment-74cf744cd6-n4tcx -- env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=gluster-s3-deployment-74cf744cd6-n4tcx
S3_ACCOUNT=adminuser
S3_USER=testuser
S3_PASSWORD=mypassword
...

test with s3curl according doc:

./s3curl.pl --debug --id "adminuser:testuser" --key "mypassword"  --put /dev/null  -- -k -v  http://10.233.57.191:8080/bucket1
WARNING: It isn't safe to put your AWS secret access key on the
command line!  The recommended key management system is to store
your AWS secret access keys in a file owned by, and only readable
by you.

For example:

%awsSecretAccessKeys = (
    # personal account
    personal => {
        id => '1ME55KNV6SBTR7EXG0R2',
        key => 'zyMrlZUKeG9UcYpwzlPko/+Ciu0K2co0duRM3fhi',
    },

    # corporate account
    company => {
        id => '1ATXQ3HHA59CYF1CVS02',
        key => 'WQY4SrSS95pJUT95V6zWea01gBKBCL6PI0cdxeH8',
    },
);

$ chmod 600 /home/vagrant/.s3curl

Will sleep and continue despite this problem.
Please set up /home/vagrant/.s3curl for future requests.
s3curl: Found the url: host=10.233.57.191; port=8080; uri=/bucket1; query=;
s3curl: cname endpoint signing case
s3curl: StringToSign='PUT\n\n\n四, 31 1月 2019 03:42:46 +0000\n/10.233.57.191/bucket1'
s3curl: exec curl -H Date: 四, 31 1月 2019 03:42:46 +0000 -H Authorization: AWS adminuser:testuser:F0T7Qxpkdm/eW7oDg1C7whjynUI= -L -H content-type:  -T /dev/null -k -v http://10.233.57.191:8080/bucket1
* About to connect() to 10.233.57.191 port 8080 (#0)
*   Trying 10.233.57.191...
* Connected to 10.233.57.191 (10.233.57.191) port 8080 (#0)
> PUT /bucket1 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.233.57.191:8080
> Accept: */*
> Transfer-Encoding: chunked
> Date: 四, 31 1月 2019 03:42:46 +0000
> Authorization: AWS adminuser:testuser:F0T7Qxpkdm/eW7oDg1C7whjynUI=
> Expect: 100-continue
> 
< HTTP/1.1 403 Forbidden
< x-amz-id-2: tx37593cc66c694810b5e64-005c526eb6
< x-amz-request-id: tx37593cc66c694810b5e64-005c526eb6
< Content-Type: application/xml
< X-Trans-Id: tx37593cc66c694810b5e64-005c526eb6
< X-Openstack-Request-Id: tx37593cc66c694810b5e64-005c526eb6
< Date: Thu, 31 Jan 2019 03:42:46 GMT
< Transfer-Encoding: chunked
* HTTP error before end of send, stop sending
< 
<?xml version='1.0' encoding='UTF-8'?>
* Closing connection 0
<Error><Code>AccessDenied</Code><Message>AWS authentication requires a valid Date or x-amz-date header</Message><RequestId>tx37593cc66c694810b5e64-005c526eb6</RequestId></Error>
SaravanaStorageNetwork commented 5 years ago

Check whether you have updated s3curl.pl with updated endpoints as stated here: https://github.com/gluster/gluster-kubernetes/blob/master/docs/examples/gluster-s3-storage-template/README.md#s3curlpl-for-testing

In your case endpoints looks like: 10.233.57.191:8080

chenyg0911 commented 5 years ago

yes. Since I use nodeport expose the service, so I update s3curl use the nodeport as endpoint.

Saravanakumar Arumugam notifications@github.com于2019年2月1日 周五上午9:02写道：

Check whether you have updated s3curl.pl with updated endpoints as stated here:

https://github.com/gluster/gluster-kubernetes/blob/master/docs/examples/gluster-s3-storage-template/README.md#s3curlpl-for-testing

In your case endpoints looks like: 10.233.57.191:8080

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gluster/gluster-kubernetes/issues/561#issuecomment-459566884, or mute the thread https://github.com/notifications/unsubscribe-auth/APRMhEqRdMVgXZB1BxqNQba3ZRsrAyO3ks5vI5JBgaJpZM4abj-o .

chenyg0911 commented 5 years ago

The problem seems cause by date format. I also try another s3 client like awscli. Use accessid “administer:testuser “/mypassword. But I got auth failed too. YongGuang Chen yongguang.chen@gmail.com于2019年2月4日 周一上午9:23写道：

yes. Since I use nodeport expose the service, so I update s3curl use the nodeport as endpoint.

Saravanakumar Arumugam notifications@github.com于2019年2月1日 周五上午9:02写道：

Check whether you have updated s3curl.pl with updated endpoints as stated here:

https://github.com/gluster/gluster-kubernetes/blob/master/docs/examples/gluster-s3-storage-template/README.md#s3curlpl-for-testing

In your case endpoints looks like: 10.233.57.191:8080

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gluster/gluster-kubernetes/issues/561#issuecomment-459566884, or mute the thread https://github.com/notifications/unsubscribe-auth/APRMhEqRdMVgXZB1BxqNQba3ZRsrAyO3ks5vI5JBgaJpZM4abj-o .

SaravanaStorageNetwork commented 5 years ago

The problem seems cause by date format. I also try another s3 client like awscli. Use accessid “administer:testuser “/mypassword. But I got auth failed too. YongGuang Chen yongguang.chen@gmail.com于2019年2月4日 周一上午9:23写道： …

Do you mean adminuser here (administer mentioned above)?

Anyway, Could you check systemctl status in the gluster-s3 pod above ? It should have swift services running properly.

SaravanaStorageNetwork commented 5 years ago

Check whether you have updated s3curl.pl with updated endpoints as stated here: https://github.com/gluster/gluster-kubernetes/blob/master/docs/examples/gluster-s3-storage-template/README.md#s3curlpl-for-testing

In your case endpoints looks like: 10.233.57.191:8080

Could you try with IP address only (removing the port number in endpoint) ? I think this should solve the issue.

chenyg0911 commented 5 years ago

yes. Tested with IP 10.233.57.191. The error is the same. The response is corect to connect to service endpoint but failed by data format. I'll try other aws client like awscli/s3cmd. Does I should use "adminuser:testuser" as accessid, and "mypassword" as secret?

attach s3curl.perl snippet:

# begin customizing here
my @endpoints = ( '10.233.57.191');

my $CURL = "curl";

# stop customizing here

output:

s3curl: Found the url: host=10.233.57.191; port=8080; uri=/bucket1; query=;
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\n一, 11 2月 2019 02:34:02 +0000\n/bucket1'
s3curl: exec curl -H Date: 一, 11 2月 2019 02:34:02 +0000 -H Authorization: AWS adminuser:testuser:4/Uhzm7K550dA5KfoVYNtXKL1xw= -L -H content-type:  -T /dev/null -k -v http://10.233.57.191:8080/bucket1
* About to connect() to 10.233.57.191 port 8080 (#0)
*   Trying 10.233.57.191...
* Connected to 10.233.57.191 (10.233.57.191) port 8080 (#0)
> PUT /bucket1 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.233.57.191:8080
> Accept: */*
> Transfer-Encoding: chunked
> Date: 一, 11 2月 2019 02:34:02 +0000
> Authorization: AWS adminuser:testuser:4/Uhzm7K550dA5KfoVYNtXKL1xw=
> Expect: 100-continue
> 
< HTTP/1.1 403 Forbidden
< x-amz-id-2: tx2536f7ba74fa4eadbf4cb-005c60df1a
< x-amz-request-id: tx2536f7ba74fa4eadbf4cb-005c60df1a
< Content-Type: application/xml
< X-Trans-Id: tx2536f7ba74fa4eadbf4cb-005c60df1a
< X-Openstack-Request-Id: tx2536f7ba74fa4eadbf4cb-005c60df1a
< Date: Mon, 11 Feb 2019 02:34:02 GMT
< Transfer-Encoding: chunked
* HTTP error before end of send, stop sending
< 
<?xml version='1.0' encoding='UTF-8'?>
* Closing connection 0
<Error><Code>AccessDenied</Code><Message>AWS authentication requires a valid Date or x-amz-date header</Message><RequestId>tx2536f7ba74fa4eadbf4cb-005c60df1a</RequestId></Error>
chenyg0911 commented 5 years ago

The problem seems cause by date format. I also try another s3 client like awscli. Use accessid “administer:testuser “/mypassword. But I got auth failed too. YongGuang Chen yongguang.chen@gmail.com于2019年2月4日 周一上午9:23写道： …

Do you mean adminuser here (administer mentioned above)?

Anyway, Could you check systemctl status in the gluster-s3 pod above ? It should have swift services running properly.

systemctl status swift-proxy.service

● swift-proxy.service - Swift Proxy Service
   Loaded: loaded (/usr/lib/systemd/system/swift-proxy.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2019-02-11 02:12:01 UTC; 45min ago
 Main PID: 248 (python)
   CGroup: /kubepods/besteffort/pod618f3db2-2472-11e9-9488-0800278bc93f/66513518262e8e01fa2c73a81ce3bd42925bd3af29fcc6b1a3e48413e3d5c43f/system.slice/swift-proxy.service
           ├─248 /usr/bin/python /usr/bin/swift-proxy-server /etc/swift/proxy...
           └─274 /usr/bin/python /usr/bin/swift-proxy-server /etc/swift/proxy...

Feb 11 02:12:01 gluster-s3-deployment-74cf744cd6-n4tcx systemd[1]: Started Sw...
Hint: Some lines were ellipsized, use -l to show in full.
chenyg0911 commented 5 years ago

test with aws cli: aws configure

aws configure
AWS Access Key ID: adminuser:testuser
AWS Secret Access Key:  mypassword
Default region name:  us-east-1
Default output format: None

aws configure set default.s3.signature_version s3v4

aws --endpoint-url http://10.233.57.191:8080 s3 ls

An error occurred (SignatureDoesNotMatch) when calling the ListBuckets operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

What's the problem? When use swift, how to configure the aws cli? Thanks!

SaravanaStorageNetwork commented 5 years ago

attach s3curl.perl snippet:

yes. Tested with IP 10.233.57.191. The error is the same. The response is corect to connect to service endpoint but failed by data format. I'll try other aws client like awscli/s3cmd. Does I should use "adminuser:testuser" as accessid, and "mypassword" as secret?

attach s3curl.perl snippet:

# begin customizing here
my @endpoints = ( '10.233.57.191');

my $CURL = "curl";

# stop customizing here

In addition to the above changes, carry out additional change:

You need to comment out:

my $htDate = POSIX::strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime);

and replace it with

my $httpDate = time2str();

Also, add the below line at the top of the file:

use HTTP::Date;

Try this change and report back.

chenyg0911 commented 5 years ago

attach s3curl.perl snippet:

yes. Tested with IP 10.233.57.191. The error is the same. The response is corect to connect to service endpoint but failed by data format. I'll try other aws client like awscli/s3cmd. Does I should use "adminuser:testuser" as accessid, and "mypassword" as secret? attach s3curl.perl snippet:

# begin customizing here
my @endpoints = ( '10.233.57.191');

my $CURL = "curl";

# stop customizing here

In addition to the above changes, carry out additional change:

You need to comment out:

my $htDate = POSIX::strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime);

and replace it with

my $httpDate = time2str();

Also, add the below line at the top of the file:

use HTTP::Date;

Try this change and report back.

change and test. s3curl test OK! Thanks @SaravanaStorageNetwork

But how can I use another s3 client like aws cli? Which AWSAcessKeyId and Secret should I use to aceess s3 service？The question I mention above. Or, Should I close the issue and open a new issue?

SaravanaStorageNetwork commented 5 years ago

change and test. s3curl test OK! Thanks @SaravanaStorageNetwork

Thank you for reporting back!

But how can I use another s3 client like aws cli?

I have not tested with s3 client like aws cli. unlike s3curl.pl I am not sure whether it supports editing endpoints.

Which AWSAcessKeyId and Secret should I use to aceess s3 service？The question I mention above. Or, Should I close the issue and open a new issue?

You could try similar to this and check. Please close this issue. Thanks!