search

google-code-export / s3ql

Automatically exported from code.google.com/p/s3ql
0 stars 0 forks source link

Support Keystone Authentication for Swift Backend #413

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi

I have trouble with s3ql  and openstack ( swift + keystone) 
Swift and keystone are working.

 swift -V 2 -A http://192.168.122.100:5000/v2.0 -U openstack:admin -K nova list
abc
denis_bucket
mountbit-s3-test
newback
newback2

 pwd
/root/.s3ql

[root@swift-02 .s3ql]# cat authinfo2
[swift]
storage-url: swift://192.168.122.100:8080/abc
backend-login: openstack:admin
backend-password: nova

mount.s3ql  --debug all  --no-ssl swift://192.168.122.100:8080/abc /mnt/abc

Using 2 upload threads.
options.storage_url = swift://192.168.122.100:8080/abc
backend_login = openstack:admin
backend_passphrase = nova
ssl_context = None
Uncaught top-level exception:
Traceback (most recent call last):
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/backends/common.py", line 1384, in get_backend_factory
    raise QuietError('No permission to access backend.') from None
s3ql.logging.QuietError: No permission to access backend.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/mount.s3ql", line 9, in <module>
    load_entry_point('s3ql==2.3', 'console_scripts', 'mount.s3ql')()
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/mount.py", line 86, in main
    backend_factory = get_backend_factory(options)
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/backends/common.py", line 1396, in get_backend_factory
    backend.close()
UnboundLocalError: local variable 'backend' referenced before assignment

swift-proxy.conf

[pipeline:main]
pipeline = catch_errors healthcheck cache ratelimit authtoken swift3 s3token  
keystone container-quotas  account-quotas proxy-logging  proxy-server

Please, why it doesn't work?  What I do incorrect?

 It works if i use   tempauth instead keystone authorization like  ( swift-proxy.conf) 

pipeline = catch_errors  cache tempauth proxy-server

 and 

[filter:tempauth]
use = egg:swift#tempauth
user_admin_admin = admin .reseller_admin .admin

Original issue reported on code.google.com by oil...@asdco.ru on 30 Jul 2013 at 7:04

GoogleCodeExporter commented 9 years ago 
Don't worry about the "During handling of the above exception, another 
exception occurred:" part, that's already fixed.

Regarding the authentication problem: S3QL doesn't currently support keystone 
authentication for the swift backend. Is tempauth the same as traditional, v1 
authentication? That would explain why that works.

If you could provide me with a test swift server, I could probably also look 
into adding keystone support to swift. The code already exists for Rackspace, 
so it shouldn't be too hard..

Original comment by Nikolaus@rath.org on 30 Jul 2013 at 4:09

GoogleCodeExporter commented 9 years ago 
tempauth is not same v1 authentication. it use another middleware.  currently  
v1 authorization works on proxy server.

sorry, i couldn't provide you with a test swift server, i will try to find it.

if I use rackspace backend  to swift, then there are errors:

 mount.s3ql  --debug all  --no-ssl rackspace://192.168.122.100:8080/abc /mnt/abc
Using 2 upload threads.
Enter backend login:
Enter backend passphrase:
options.storage_url = rackspace://192.168.122.100:8080/abc
backend_login = openstack:admin
backend_passphrase = nova
ssl_context = None
Uncaught top-level exception:
Traceback (most recent call last):
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/backends/common.py", line 1373, in get_backend_factory
    backend = backend_class(options.storage_url, backend_login, backend_passphrase, ssl_context)
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/backends/rackspace.py", line 32, in __init__
    (region, container_name, prefix) = self._parse_storage_url(storage_url)
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/backends/rackspace.py", line 63, in _parse_storage_url
    raise QuietError('Invalid storage URL')
s3ql.logging.QuietError: Invalid storage URL

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/mount.s3ql", line 9, in <module>
    load_entry_point('s3ql==2.3', 'console_scripts', 'mount.s3ql')()
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/mount.py", line 86, in main
    backend_factory = get_backend_factory(options)
  File "/usr/lib64/python3.3/site-packages/s3ql-2.3-py3.3-linux-x86_64.egg/s3ql/backends/common.py", line 1396, in get_backend_factory
    backend.close()
UnboundLocalError: local variable 'backend' referenced before assignment

Original comment by oil...@asdco.ru on 31 Jul 2013 at 1:03

GoogleCodeExporter commented 9 years ago 
The RackSpace backend is for Rackspace (www.rackspace.com), and not any swift 
server. The storage URL you specify is invalid, cf. 
http://www.rath.org/s3ql-docs/backends.html#rackspace-cloudfiles

Original comment by Nikolaus@rath.org on 31 Jul 2013 at 3:53

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
I created swift test stand with credentials:
If you want you can   look into adding keystone support to swift.  I have sent 
it on your email.

Original comment by oil...@asdco.ru on 1 Aug 2013 at 12:03

GoogleCodeExporter commented 9 years ago 
Do you have some link that describes how keystone authentication should be 
implemented for Swift?

When I try the procedure described for Rackspace 
(http://docs.rackspace.com/auth/api/v2.0/auth-client-devguide/content/Sample_Req
uest_Response-d1e64.html), your test server just responds with either:

POST /v2.0/tokens HTTP/1.1
Host: xxx
Accept-Encoding: identity
Content-Length: 112
Content-Type: application/json
Accept: application/json; charset="utf-8"

{"auth": {"tenantId": "xxxx", "passwordCredentials": {"password": "xxxx", 
"username": "xxx"}}}

HTTP/1.1 401 Not Authorized
Vary: X-Auth-Token
Content-Type: application/json
Content-Length: 81
Date: Sat, 03 Aug 2013 04:27:01 GMT

{"error": {"message": "Invalid project", "code": 401, "title": "Not 
Authorized"}}

Or, alternatively, if I omit the tenant:

POST /v2.0/tokens HTTP/1.1
Host: xxx
Accept-Encoding: identity
Content-Length: 85
Content-Type: application/json
Accept: application/json; charset="utf-8"

{"auth": {"passwordCredentials": {"password": "xxx", "username": "xxx"}}}

HTTP/1.1 200 OK
Vary: X-Auth-Token
Content-Type: application/json
Content-Length: 337
Date: Sat, 03 Aug 2013 04:31:23 GMT

{"access": {"token": {"issued_at": "2013-08-03T04:31:23.797781", "expires": 
"2013-08-04T04:31:23Z", "id": "1f497xxxxxxb51b524aa746f7b6"}, "serviceCatalog": 
[], "user": {"username": "xxx", "roles_links": [], "id": 
"5194c3xxxxcfe96ebf6f941", "roles": [], "name": "s3ql-user"}, "metadata": 
{"is_admin": 0, "roles": []}}}

In either case, it does not list any storage services.

Original comment by Nikolaus@rath.org on 3 Aug 2013 at 4:32

GoogleCodeExporter commented 9 years ago 
Nevermind, I just found 
http://docs.openstack.org/api/openstack-identity-service/2.0/content/. It seems 
the tenant has to be specified as tenantName rather than tenantId.

Do you have any thoughts on how the user should select the proper endpoint? I'm 
tempted to include the region in the storage URL (as done in the rackspace 
backend).

Original comment by Nikolaus@rath.org on 3 Aug 2013 at 4:49

GoogleCodeExporter commented 9 years ago 
This issue was closed by revision 823398c682b0.

Original comment by Nikolaus@rath.org on 4 Aug 2013 at 12:55