Cannot use swiftly for openstack swift

[bimboh8]

We are very happy using the swiftly CLI for rackspace to manage backup's. Rackspace recommends your module which is like a big yaay for you.

We have had issues using the same for our DC which is based on vanilla openstack. Error : swiftly --conf=/etc/swiftly/swiftly.conf get --output=node_container.json Container_name

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/cli.py", line 460, in _perform_command
    self.commands[command](args[1:])
  File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/get.py", line 514, in __call__
    return cli_get(context, path)
  File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/get.py", line 285, in cli_get
    return cli_get_container_listing(context, path)
  File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/get.py", line 203, in cli_get_container_listing
    with context.client_manager.with_client() as client:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 75, in with_client
    client = self.get_client()
  File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 56, in get_client
    client = self.client_class(*self.args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/swiftly/client/directclient.py", line 79, in __init__
    import swift.proxy.server

Following is the configuration file we are using -

# Config file for swiftly client.
[swiftly]
#The URL to the auth system, example:
auth_url: Openstack URL

#The user name for the auth system, example: test:tester
auth_user: User Information for DC

#The key for the auth system, example: testing
auth_key: We use password instead of token. Is this a problem

#Uses direct connect method to access Swift. Requires access to rings and
#backend servers. The PATH is the account path, example: /v1/AUTH_test
direct = Direct Access to the API

#Uses the given HTTP proxy URL.
proxy = No proxy

#Default: default region specified by the auth response.
region: The region of the DC

#Indicates how many times to retry the request on a server error. Default: 4
retries: 4

#Causes output to standard error indicating actions being taken. These
#output lines will be prefixed with VERBOSE and will also include the number
#of seconds elapsed since the command started.
verbose: False

#direct_object_ring = We dont need this since its direct access
#Custom object ring to be used in direct connect method to access Swift.
#The PATH is the custom object ring file path, 
#example: /etc/swift/custom-object.ring.gz

Questions -

1. We have not setup tokens for auth_key? Is this an issue that it only works with tokens.
2. Although the error says that's it cannot load up module it works for rackspace containers.

Regards bimboh

[gholt]

I'm not sure what the error was. It may have got lost in the post. I see the traceback but not the actual error message.

[gholt]

Also, if you're using direct access auth should not be a problem, as the Swift proxy server is what needs auth and you're bypassing that and going directly to the backend servers.

[gholt]

Ah, I see maybe what is getting you. I forgot that Python stack traces have most recent last (I've gotten used to Go which does it opposite).

Anyway, it looks like you don't have the Swift codebase installed where you are running Swiftly. You need that codebase if you're going to use direct mode.

Direct mode essentially loads up Swift's proxy server code and uses that to connect directly to the backend servers.

However.... I have not used that feature in quite some time, so it could be giving a different error due to changes in the Swift codebase since I last tested the feature.

If you do have the Swift codebase installed where you are running Swiftly, can you run your command again and repaste the full error? It seems the actual error message got snipped from your report.

Thanks!

[bimboh8]

Thank you for the suggestions.

Apologies for delayed response. Here is the full stack trace that i get in full. Its very hard to debug the issue with this much information

Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/cli.py", line 460, in _perform_command self.commandscommand File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/get.py", line 514, in call return cli_get(context, path) File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/get.py", line 285, in cli_get return cli_get_container_listing(context, path) File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/get.py", line 203, in cli_get_container_listing with context.client_manager.with_client() as client: File "/usr/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 75, in with_client client = self.get_client() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 56, in get_client client = self.client_class(*self.args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/swiftly/client/directclient.py", line 79, in init import swift.proxy.server ImportError: No module named swift.proxy.server

I tried installing swift code base and the thing that gets me is that when i do the same thing via swift cli it works.I get back the container and i can perform push/pull from object store.

**swift --os-auth-url <auth-url> --os-project-name <project-name> --os-project-domain-name <domain-name> --os-username <username> --os-password <password> list**

Is it possible that swiftly uses auth_key and we probably might have set token or authentication. In rackspace we can generate a key. But i'm confused because if i do the same via swift CLI i can still use via the password.

I somehow feel that the issue is caused when we use the direct access to swift. Thanks!

[gholt]

I think you just might have a "bad" config. Try renaming the config to donotuserightnow or something and just use command line arguments and see if that heps. e.g. swiftly -A <authurl> -U <domainname:username> -K <password> stat

Unless you are trying to run in "direct mode"? Most people do not use that mode except for in advanced operations situations. Direct mode means that swiftly will pretend it is the Swift Proxy server and it will not use auth at all and will directly connect to Swift Account, Container, and Object servers.

[bimboh8]

Thank you for your valuable suggestion, I tried the following options.

1. Changing the file did not help, i get the same proxy error.

2. swiftly -A authurl -U domainname:username -K password stat Resulted in ERROR unknown command 'stat'

3. swiftly -A authurl -U domainname:username -K password auth

Resulted in Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/cli.py", line 460, in _perform_command self.commandscommand File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 140, in call return cli_auth(context) File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 43, in cli_auth client.auth() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/standardclient.py", line 233, in auth status, reason = func() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/standardclient.py", line 299, in _auth2key return self._auth2('RAX-KSKEY:apiKeyCredentials') File "/usr/local/lib/python2.7/dist-packages/swiftly/client/standardclient.py", line 329, in _auth2 'User-Agent': self.user_agent}) File "/usr/lib/python2.7/httplib.py", line 1039, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1073, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1035, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 879, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 841, in send self.connect() File "/usr/lib/python2.7/httplib.py", line 1250, in connect server_hostname=server_hostname) File "/usr/lib/python2.7/ssl.py", line 350, in wrap_socket _context=self) File "/usr/lib/python2.7/ssl.py", line 560, in init server_hostname, ssl_sock=self) TypeError: _wrap_socket() argument 1 must be _socket.socket, not GreenSocket

4. swiftly -A authurl -D directpath -U domainname:username -K password auth

Resulted in Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/cli.py", line 460, in _perform_command self.commandscommand File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 140, in call return cli_auth(context) File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 41, in cli_auth context.client_manager.with_client() as client: File "/usr/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 75, in with_client client = self.get_client() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 56, in get_client client = self.client_class(*self.args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/swiftly/client/directclient.py", line 79, in init import swift.proxy.server ImportError: No module named swift.proxy.server

Thanks!

[gholt]

Ah, I forgot stat is a swift command and not for swiftly. Sorry about that, my memory is rusty. :)

3. Looks like what you should be doing, but I can't figure out why you're getting that GreenSocket error.

I did some Google searching on TypeError: _wrap_socket() argument 1 must be _socket.socket, not GreenSocket and just came up with it might be an older version of eventlet installed.

Just for reference, I ran the following on my server here:

$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import eventlet
>>> eventlet.__version__
'0.18.4'
>>> 

[bimboh8]

Thanks , No problem i also usually mix up the commands of different cli's. I tried upgrading the eventlet to the version you had.

Python 2.7.9 (default, Jun 29 2016, 13:08:31) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import eventlet eventlet.version '0.18.4'

swiftly -A auth_url -U domainname:username -K password auth

Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/cli.py", line 460, in _perform_command self.commandscommand File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 140, in call return cli_auth(context) File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 43, in cli_auth client.auth() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/standardclient.py", line 238, in auth raise self.HTTPException('Auth failure %r.' % info) HTTPException: Auth failure ['404 Not Found', '404 Not Found', '404 Not Found', '0 No x-storage-url header'].

So it says no-storage_url which probably is referring to the direct access. I tried adding and it gives me the same proxy error

swiftly -A auth_url -D direct_path -U domainname:username -K password auth

Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/cli.py", line 460, in _perform_command self.commandscommand File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 140, in call return cli_auth(context) File "/usr/local/lib/python2.7/dist-packages/swiftly/cli/auth.py", line 41, in cli_auth context.client_manager.with_client() as client: File "/usr/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 75, in with_client client = self.get_client() File "/usr/local/lib/python2.7/dist-packages/swiftly/client/manager.py", line 56, in get_client client = self.client_class(*self.args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/swiftly/client/directclient.py", line 79, in init import swift.proxy.server ImportError: No module named swift.proxy.server

Thanks!

[bimboh8]

Finally Some Good news. The issue was with the api_url and it was not working with the one exposed internally. I had to go to an older api version to make it work. Thanks a lot for all the effort you put in to help me. It also worked with the config as well. We can close the issue.

Thanks.

[gholt]

\o/ Good deal! I'm glad you figured it out. Sometimes these things can be quite maddening.