search

keijack / python-eureka-client

A eureka client written in python. Support registering your python component to Eureka Server, as well as calling remote services by pulling the the Eureka registry.
MIT License
183 stars 43 forks source link

Broken Eureka Client 0.8.2 / 0.8.3 / 0.8.4 #37

Closed dinuta closed 3 years ago

dinuta commented 3 years ago

Hi man,

The latest builds are broken, or you changed the functionality.

Consider the tests found at https://github.com/dinuta/netflixoss-eureka/tree/master/tests. The docker compose is in the same repository.

There is the following scenario:

  1. There are 2 eureka that form a cluster.
  2. A microservice which implements your client connects to this cluster.
  3. One instance or the other or both come down and then recovers.

Expected result: This microservice should appear registered on the both eurekas, when one of them recovers.

Actual result: The microservice is not present on the instance that recovered.

Latest build that worked: 0.7.4

keijack commented 3 years ago

I have check your test cases, I cannot find the microserivce registration code. Could you please tell where it is? Or have you changed the implementation? I have only find out that you use the get_application method to load the registry.

dinuta commented 3 years ago

Hi @keijack ,

To reproduce you have here the code: https://github.com/dinuta/estuary-discovery in the main_flask.py. Just start the service having these 3 env vars:

To register to the both eureka use the following env var settings: EUREKA_SERVER=yourcluster. E.g. EUREKA_SERVER: "http://localhost:8080/eureka/v2,http://localhost:8081/eureka/v2" APP_IP_PORT=your_app_ip:your_app_port. E.g. 127.0.0.1:8090 PORT=8090

You can use that docker-compose to get the eureka servers on docker hub and forward the ports on your machine. Then configure the service above with different eureka_client in requirements.txt.

Don't forget to pip install the versions you test.

dinuta commented 3 years ago

Hi again @keijack

Seems like bad things happen on the latest builds. This is stdout executed in the same conditions. Docker-compose.yml:

version: "3.3"

services:
  eureka-server1:
    image: dinutac/netflixoss-eureka:latest
    environment:
      JAVA_OPTS: "-Deureka.serviceUrl.default=http://eureka-server2:8080/eureka/v2/"
    hostname: eureka-server1
    ports:
      - "8080:8080"
    expose:
      - "8080"

  # one eureka server points to the other one

  eureka-server2:
    image: dinutac/netflixoss-eureka:latest
    hostname: eureka-server2
    environment:
      JAVA_OPTS: "-Deureka.serviceUrl.default=http://eureka-server1:8080/eureka/v2/"
    ports:
      - "8081:8080"
    expose:
      - "8080"

Bring down and then up an eureka instance after initially both are up and discovery registers:

docker-compose stop eureka-server1
docker-compose start eureka-server1

With client 0.7.4:

(Thread-1  ) Eureka server [http://localhost:8080/eureka/v2] is down, use next url to try.
[2020-10-16 09:39:24]-[EurekaClient]-[line:647] -WARNING: Eureka server [http://localhost:8080/eureka/v2] is down, use next url to try.

With client 0.8.4:

[2020-10-16 09:41:51]-[eureka_client]-[line:1085] -ERROR: Eureka server [http://localhost:8080/eureka/v2] is down, use next url to try.
Traceback (most recent call last):
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1319, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
    self.send(msg)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
    self.connect()
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 938, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\socket.py", line 728, in create_connection
    raise err
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 1099, in __connect_to_eureka_server
    self.__try_eureka_server_in_cache(fun)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 1041, in __try_eureka_server_in_cache
    fun(url)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 1174, in <lambda>
    status=self.__instance["status"], overriddenstatus=overridden_status))
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 377, in status_update
    http_client.load(req, timeout=_DEFAULT_TIME_OUT)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\http_client.py", line 152, in load
    cafile=cafile, capath=capath, cadefault=cadefault, context=context).urlopen()
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\http_client.py", line 117, in urlopen
    cadefault=self.cadefault, context=self.context)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 543, in _open
    '_open', req)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1347, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1321, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1319, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
    self.send(msg)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
    self.connect()
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 938, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\socket.py", line 728, in create_connection
    raise err
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 1083, in __try_eureka_servers_in_list
    fun(url)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 1174, in <lambda>
    status=self.__instance["status"], overriddenstatus=overridden_status))
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\eureka_client.py", line 377, in status_update
    http_client.load(req, timeout=_DEFAULT_TIME_OUT)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\http_client.py", line 152, in load
    cafile=cafile, capath=capath, cadefault=cadefault, context=context).urlopen()
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\site-packages\py_eureka_client\http_client.py", line 117, in urlopen
    cadefault=self.cadefault, context=self.context)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 543, in _open
    '_open', req)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1347, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Users\Catalin Dinuta\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1321, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>
dinuta commented 3 years ago

I have check your test cases, I cannot find the microserivce registration code. Could you please tell where it is? Or have you changed the implementation? I have only find out that you use the get_application method to load the registry.

https://github.com/dinuta/estuary-discovery/blob/master/rest/api/eureka_registrator.py

keijack commented 3 years ago

OK, I will check it out, but the logs you provide it not what cause the error, 0.8.4 will output the error logs where 0.7.4 hides it and only give a warning. Will your program exist when finding an "error" log?

dinuta commented 3 years ago

Hi man,

Did you fixed the build in 0.8.5? Should I try?

keijack commented 3 years ago

I've just make the ERROR logs to be WARNING in 0.8.5, the main cause of this problem is still not being found.

dinuta commented 3 years ago

Hi @keijack ,

The solution is to incrementally test from version 0.7.4 and see where the things were broken.

dinuta commented 3 years ago

HI @keijack ,

The client was broken when you upgraded from 0.7.4 to 0.7.5. Please investigate the changes between these 2 versions.

I really need a newer version with newer features for my production environment.

Catalin

keijack commented 3 years ago

According to the release history, 0.7.5 fixed issue #24 , the deprecated url apps/%s/%s?status=%s&lastDirtyTimestamp=%s was changed to apps/%s/%s/status?value=%s&lastDirtyTimestamp=%s, I've test that in both Netfix Eureka Server and Spring Eureka, However, I've read the article, and use the apps/%s/%s?status=%s&lastDirtyTimestamp=%s for heartbeat and apps/%s/%s/status?value=%s&lastDirtyTimestamp=%s for status updating. 0.8.6 should work, please check.

dinuta commented 3 years ago

Hi @keijack ,

Tested on 1.10.5 (the latest is 1.10.8). Everything is fine !!!

Nice work man, and thank you.

Maybe you can integrate some of my tests into yours (or create something similar), before releasing new versions to not break anything in the future.

Catalin