search

openvstorage / framework

The Framework is a set of components and tools which brings the user an interface (GUI / API) to setup, extend and manage an Open vStorage platform.
Other
27 stars 23 forks source link

EOFError during request to 127.0.0.1 #925

Closed kinvaris closed 8 years ago

kinvaris commented 8 years ago

Some requests are throwing not authenticated or EOFError exceptions due the problem that 127.0.0.1 or localhost is not known to the system in .ssh/known_hosts:

RuntimeError: Authentication failed. (internal_server)
2016-09-19 13:47:38 94300 +0200 - ctrl-roub-01 - 7527/140131513165296 - api/middleware - 5593 - ERROR - An unhandled exception occurred: 
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/rest_framework/viewsets.py", line 78, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/csrf.py", line 57, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/rest_framework/views.py", line 399, in dispatch
    response = self.handle_exception(exc)
  File "/usr/lib/python2.7/dist-packages/rest_framework/views.py", line 396, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/OpenvStorage/webapps/api/backend/decorators.py", line 461, in new_function
    return_value = f(*args, **kwargs)
  File "/opt/OpenvStorage/webapps/api/backend/decorators.py", line 79, in new_function
    return f(*args, **kw)
  File "/opt/OpenvStorage/webapps/api/backend/decorators.py", line 322, in new_function
    return Response(FullSerializer(object_type, contents=contents, instance=obj).data, status=status.HTTP_200_OK)
  File "/usr/lib/python2.7/dist-packages/rest_framework/serializers.py", line 573, in data
    self._data = self.to_native(obj)
  File "/usr/lib/python2.7/dist-packages/rest_framework/serializers.py", line 349, in to_native
    value = field.field_to_native(obj, field_name)
  File "/usr/lib/python2.7/dist-packages/rest_framework/fields.py", line 198, in field_to_native
    value = get_component(value, component)
  File "/usr/lib/python2.7/dist-packages/rest_framework/fields.py", line 56, in get_component
    val = getattr(obj, attr_name)
  File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 341, in <lambda>
    fget = lambda s: s._get_dynamic_property(dynamic)
  File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 410, in _get_dynamic_property
    return self._backend_property(data_loader, dynamic)
  File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 904, in _backend_property
    dynamic_data = function()
  File "/opt/OpenvStorage/ovs/dal/hybrids/albabackend.py", line 507, in _local_summary
    for node_values in self.local_stack.itervalues():
  File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 341, in <lambda>
    fget = lambda s: s._get_dynamic_property(dynamic)
  File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 410, in _get_dynamic_property
    return self._backend_property(data_loader, dynamic)
  File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 904, in _backend_property
    dynamic_data = function()
  File "/opt/OpenvStorage/ovs/dal/hybrids/albabackend.py", line 163, in _local_stack
    config = Configuration.get_configuration_path('/ovs/arakoon/{0}/config'.format(self.abm_services[0].service.name))
  File "/opt/OpenvStorage/ovs/extensions/generic/configuration.py", line 101, in get_configuration_path
    key=key)
  File "/opt/OpenvStorage/ovs/extensions/generic/configuration.py", line 404, in _passthrough
    return getattr(ArakoonConfiguration, method)(*args, **kwargs)
  File "/opt/OpenvStorage/ovs/extensions/db/arakoon/configuration.py", line 71, in get_configuration_path
    config.load_config('127.0.0.1')
  File "/opt/OpenvStorage/ovs/extensions/db/arakoon/ArakoonInstaller.py", line 124, in load_config
    contents = client.file_read(self.config_path)
  File "/opt/OpenvStorage/ovs/extensions/generic/sshclient.py", line 489, in file_read
    return self.run('cat "{0}"'.format(filename))
  File "/opt/OpenvStorage/ovs/extensions/generic/sshclient.py", line 59, in inner_function
    return outer_function(self, *args, **kwargs)
  File "/opt/OpenvStorage/ovs/extensions/generic/sshclient.py", line 248, in run
    _, stdout, stderr = self._client.exec_command(command)  # stdin, stdout, stderr
  File "/usr/lib/python2.7/dist-packages/paramiko/client.py", line 370, in exec_command
    chan = self._transport.open_session()
  File "/usr/lib/python2.7/dist-packages/paramiko/transport.py", line 662, in open_session
    return self.open_channel('session')
  File "/usr/lib/python2.7/dist-packages/paramiko/transport.py", line 764, in open_channel
    raise e
EOFError

As seen in /var/log/auth.log

Sep 19 13:47:38 ctrl-roub-01 sshd[78353]: Accepted publickey for ovs from 127.0.0.1 port 33462 ssh2: RSA 8c:90:a0:e0:cb:fb:df:86:f9:61:57:fe:be:0d:af:b6
Sep 19 13:47:38 ctrl-roub-01 sshd[78353]: pam_unix(sshd:session): session opened for user ovs by (uid=0)
Sep 19 13:47:38 ctrl-roub-01 sshd[78353]: dispatch_protocol_error: type 90 seq 3 [preauth]

  File "/opt/OpenvStorage/ovs/extensions/db/arakoon/configuration.py", line 71, in get_configuration_path
    config.load_config('127.0.0.1')

As proven here:

root -> ovs

root@ctrl-stras-03:~# ssh ovs@127.0.0.1
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ED25519 key fingerprint is 6e:cc:fd:c9:60:4c:34:f5:17:38:c4:7a:2f:3e:18:75.
Are you sure you want to continue connecting (yes/no)? 

ovs -> ovs

ovs@ctrl-stras-03:~# ssh ovs@127.0.0.1
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ED25519 key fingerprint is 6e:cc:fd:c9:60:4c:34:f5:17:38:c4:7a:2f:3e:18:75.
Are you sure you want to continue connecting (yes/no)? 

ovs -> root

ovs@perf-roub-01:~$ ssh root@127.0.0.1
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ED25519 key fingerprint is 05:ae:4d:81:d5:ab:1b:37:1a:c7:78:d4:7b:95:a3:fa.
Are you sure you want to continue connecting (yes/no)? 

root -> root

root@perf-roub-01:~# ssh root@127.0.0.1
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ED25519 key fingerprint is 05:ae:4d:81:d5:ab:1b:37:1a:c7:78:d4:7b:95:a3:fa.
Are you sure you want to continue connecting (yes/no)?
khenderick commented 8 years ago

Apparently we shouldn't use 127.0.0.1 but we should figure out the effective ip address. Probably the test env was too small to hit this issue.

khenderick commented 8 years ago

Fixed by #931 and openvstorage/alba-asdmanager#114, packaged in openvstorage-2.7.3-rev.3992.df37d92 and openvstorage-sdm-1.6.3-rev.361.e7bf658

JeffreyDevloo commented 8 years ago

Information

We encountered this error on OVH and it was hot fixed when we reported the problem. Since the patch, we did not encounter this error anymore on the same environment.

This problem was not reported on our testing builds as well.

Steps

Output

Could process the normal flow without any errors occurring (meaning that the Authentication Error did not get thrown)

Test results

Due to the fact that the patch was created as a necessisity to install our environment and the environmnet has been succesfully running afterwards, I conclude that this error has been resolved.

A code analysis concluded that the usage of localhost is still used on three points:

Test passed.

Packages