search

ElasticHQ / elasticsearch-HQ

Monitoring and Management Web Application for ElasticSearch instances and clusters.
http://www.elastichq.org
Other
4.96k stars 530 forks source link

RecursionError: maximum recursion depth exceeded while calling a Python object #500

Closed jerome83136 closed 2 years ago

jerome83136 commented 4 years ago

General information

Hello,

I'm trying to configure Elastic-HQ to monitor my Elastic stack.

  1. I launch it the following way: python3.6 application.py --enable-ssl --verify_certs=false

  2. It starts with no errors. Then I go to http://MYHQSERVER:5000

  3. I then try to access to one of my Elastic master nodes this by entering: https://**LOGIN**:**PASSWORD**@**MYESMASTERNODE*:**PORT**

  4. I have nothing in the browser and I get these logs: `2019-10-22 00:07:43,082 INFO engineio server.init:140 Server initialized for eventlet. None None --- Logging error --- Traceback (most recent call last): File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 70, in create_connection timeout=REQUEST_TIMEOUT, verify=ca_certs, cert=client_cert_credentials) File "/usr/lib/python3.6/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, kwargs) File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request self._validate_conn(conn) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn conn.connect() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect cert_reqs=resolve_cert_reqs(self.certreqs), File "/usr/lib/python3.6/site-packages/urllib3/util/ssl.py", line 269, in create_urllib3_context context.options |= options File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) [Previous line repeated 295 more times] RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib64/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib64/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/usr/lib64/python3.6/logging/init.py", line 577, in format record.message = record.getMessage() File "/usr/lib64/python3.6/logging/init.py", line 338, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 221, in main result = function(args, kwargs) File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request proto.init(conn_state, self) File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 357, in init self.handle() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 390, in handle self.handle_one_request() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 466, in handle_one_request self.handle_one_response() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 566, in handle_one_response result = self.application(self.environ, start_response) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1997, in call return self.wsgi_app(environ, start_response) File "/usr/lib/python3.6/site-packages/flask_socketio/init.py", line 43, in call start_response) File "/usr/lib/python3.6/site-packages/engineio/middleware.py", line 67, in call return self.wsgi_app(environ, start_response) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request return self.view_functions[rule.endpoint](req.view_args) File "/usr/lib64/python3.6/site-packages/flask_restful/init.py", line 480, in wrapper resp = resource(args, kwargs) File "/usr/lib/python3.6/site-packages/flask/views.py", line 84, in view return self.dispatch_request(*args, *kwargs) File "/usr/lib64/python3.6/site-packages/flask_restful/init.py", line 595, in dispatch_request resp = meth(args, kwargs) File "/products/elasticsearch-HQ/elastichq/common/exceptions.py", line 29, in _request_wrapper return functor(*args, kwargs) File "/products/elasticsearch-HQ/elastichq/api/clusters.py", line 129, in post client_cert=client_cert) File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 126, in create_connection LOG.error(message, ex) Message: 'Unable to create connection to: https://MYESMASTERNODE:PORT**' Arguments: (RecursionError('maximum recursion depth exceeded while calling a Python object',),) ` NB:

How can I fixe this ?

Thank you for your help

Best regards

royrusso commented 4 years ago

Just out of curiosity, why did you not specify a path to the cert? --ca-certs /path/to/your/ca.crt See here: http://docs.elastichq.org/installation.html#connecting-with-ssl

jerome83136 commented 4 years ago

I thought that providing the CA was optionnal when verify_certs=false

I also tried putting a path to the CA cert this way:

python3.6 application.py --enable-ssl --verify_certs=false --ca-certs=/path/to/my/ca/cert/mycacert.crt

but got the same result:

`2019-10-22 09:52:52,245 INFO engineio server.init:140 Server initialized for eventlet. None None --- Logging error --- Traceback (most recent call last): File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 70, in create_connection timeout=REQUEST_TIMEOUT, verify=ca_certs, cert=client_cert_credentials) File "/usr/lib/python3.6/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, kwargs) File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request self._validate_conn(conn) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn conn.connect() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect cert_reqs=resolve_cert_reqs(self.certreqs), File "/usr/lib/python3.6/site-packages/urllib3/util/ssl.py", line 269, in create_urllib3_context context.options |= options File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) [Previous line repeated 295 more times] RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib64/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib64/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/usr/lib64/python3.6/logging/init.py", line 577, in format record.message = record.getMessage() File "/usr/lib64/python3.6/logging/init.py", line 338, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 221, in main result = function(args, kwargs) File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request proto.init(conn_state, self) File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 357, in init self.handle() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 390, in handle self.handle_one_request() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 466, in handle_one_request self.handle_one_response() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 566, in handle_one_response result = self.application(self.environ, start_response) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1997, in call return self.wsgi_app(environ, start_response) File "/usr/lib/python3.6/site-packages/flask_socketio/init.py", line 43, in call start_response) File "/usr/lib/python3.6/site-packages/engineio/middleware.py", line 67, in call return self.wsgi_app(environ, start_response) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request return self.view_functions[rule.endpoint](req.view_args) File "/usr/lib64/python3.6/site-packages/flask_restful/init.py", line 480, in wrapper resp = resource(args, kwargs) File "/usr/lib/python3.6/site-packages/flask/views.py", line 84, in view return self.dispatch_request(*args, *kwargs) File "/usr/lib64/python3.6/site-packages/flask_restful/init.py", line 595, in dispatch_request resp = meth(args, kwargs) File "/products/elasticsearch-HQ/elastichq/common/exceptions.py", line 29, in _request_wrapper return functor(*args, kwargs) File "/products/elasticsearch-HQ/elastichq/api/clusters.py", line 129, in post client_cert=client_cert) File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 126, in create_connection LOG.error(message, ex) Message: 'Unable to create connection to: https://MYESMASTERNODE:PORT' Arguments: (RecursionError('maximum recursion depth exceeded while calling a Python object',),) --- Logging error --- Traceback (most recent call last): File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 70, in create_connection timeout=REQUEST_TIMEOUT, verify=ca_certs, cert=client_cert_credentials) File "/usr/lib/python3.6/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, kwargs) File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request self._validate_conn(conn) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn conn.connect() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect cert_reqs=resolve_cert_reqs(self.certreqs), File "/usr/lib/python3.6/site-packages/urllib3/util/ssl.py", line 269, in create_urllib3_context context.options |= options File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) [Previous line repeated 295 more times] RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib64/python3.6/logging/handlers.py", line 71, in emit if self.shouldRollover(record): File "/usr/lib64/python3.6/logging/handlers.py", line 187, in shouldRollover msg = "%s\n" % self.format(record) File "/usr/lib64/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/usr/lib64/python3.6/logging/init.py", line 577, in format record.message = record.getMessage() File "/usr/lib64/python3.6/logging/init.py", line 338, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 221, in main result = function(args, kwargs) File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request proto.init(conn_state, self) File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 357, in init self.handle() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 390, in handle self.handle_one_request() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 466, in handle_one_request self.handle_one_response() File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 566, in handle_one_response result = self.application(self.environ, start_response) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1997, in call return self.wsgi_app(environ, start_response) File "/usr/lib/python3.6/site-packages/flask_socketio/init.py", line 43, in call start_response) File "/usr/lib/python3.6/site-packages/engineio/middleware.py", line 67, in call return self.wsgi_app(environ, start_response) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request return self.view_functions[rule.endpoint](req.view_args) File "/usr/lib64/python3.6/site-packages/flask_restful/init.py", line 480, in wrapper resp = resource(args, kwargs) File "/usr/lib/python3.6/site-packages/flask/views.py", line 84, in view return self.dispatch_request(*args, *kwargs) File "/usr/lib64/python3.6/site-packages/flask_restful/init.py", line 595, in dispatch_request resp = meth(args, kwargs) File "/products/elasticsearch-HQ/elastichq/common/exceptions.py", line 29, in _request_wrapper return functor(*args, *kwargs) File "/products/elasticsearch-HQ/elastichq/api/clusters.py", line 129, in post client_cert=client_cert) File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 126, in create_connection LOG.error(message, ex) Message: 'Unable to create connection to: https://MYESMASTERNODE:PORT' Arguments: (RecursionError('maximum recursion depth exceeded while calling a Python object',),) 2019-10-22 09:53:00,105 ERROR elastichq exceptions._request_wrapper:37 Oops! Something bad happened. Traceback (most recent call last): File "/products/elasticsearch-HQ/elastichq/common/exceptions.py", line 29, in _request_wrapper return functor(args, kwargs) File "/products/elasticsearch-HQ/elastichq/api/clusters.py", line 129, in post client_cert=client_cert) File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 127, in create_connection raise ex File "/products/elasticsearch-HQ/elastichq/service/ConnectionService.py", line 70, in create_connection timeout=REQUEST_TIMEOUT, verify=ca_certs, cert=client_cert_credentials) File "/usr/lib/python3.6/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, kwargs) File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request self._validate_conn(conn) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn conn.connect() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 314, in connect cert_reqs=resolve_cert_reqs(self.certreqs), File "/usr/lib/python3.6/site-packages/urllib3/util/ssl.py", line 269, in create_urllib3_context context.options |= options File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) File "/usr/lib64/python3.6/ssl.py", line 465, in options super(SSLContext, SSLContext).options.set(self, value) [Previous line repeated 295 more times] RecursionError: maximum recursion depth exceeded while calling a Python object `

gbit-is commented 4 years ago

I was trying elasticsearch-HQ for the first time, fired it up on ubuntu 16.04, no problem

tried it on ubuntu 18 and then I got the error described here.

I compared the pip packages on both end, installed the specific versions of the server it worked on the one where it didn't but that did nothing

python3 versions of the 2 servers: (according to dpkg and then python --version)
Ubuntu 16: 3.5.1-3 (Python 3.5.2)
Ubuntu 18: 3.6.7-1~18.04 (Python 3.6.8)

rollinsjw commented 4 years ago

I was having the same issue, I switched to python 3.4.9 and changed the pytest requirement to 4.6.6 and I managed to get it working.

I ran elastic-HQ as such:

python3.9 application.py --enable-ssl --verify_certs=false

rschirin commented 4 years ago

Hi, I can confirm that using Python3.6 on Centos 7 I was facing the same error (I have the same initial scenario). Using python3.4 everything works correctly

jamisonstar commented 4 years ago

Hi, I stuck in this issue many days. Everything goes fine after using Python 3.4.10

Lusitaniae commented 4 years ago

Same issue,

Trying to connect to https endpoint from AWS ES cluster running 7.4.

I've tried latest, v3.5.3 and release-v3.4.0 docker images.

I've tried master from python 3.6.9 locally, doesn't work either, trying to install 3.4 but is failing due to dependencies.

Cyb3rSn0rlax commented 4 years ago

Having the same issue here with my root-ca.pem certificate on Ubuntu 18 LTS with a cluster of 3 nodes