minos-framework / minos-discovery

🔍 The Discovery service of the Minos Framework
http://www.minos.run
MIT License
7 stars 2 forks source link

[Subscription] Incorrect microservice IP #78

Open vladyslav-fenchak opened 2 years ago

vladyslav-fenchak commented 2 years ago

A microservice is restarted but for some reason it does not disconnect correctly from the discovery (it does not send the unsubscribe or the discovery is already restarting and therefore does not receive the request). later, when it tries to connect to the discovery, it says that it is already connected, so it does not update the IP assuming that it keeps the same one (that would be the bug). And after restarting the microservice once again, the unsubscribe is done. And then, after restarting the microservice again, it does the subscribe again, and in this case it registers the correct IP of the microservice.

garciparedes commented 2 years ago

Here is a theory about the bug reason:

This is the handling function that processes the subscription calls from microservices:

https://github.com/minos-framework/minos-discovery/blob/87f8cf8aa5229864360a9a4333f19915b303f02f/minos/api_gateway/discovery/views/microservice.py#L22-L31

Here is the save method from Microservice, that performs the set_data calls.

https://github.com/minos-framework/minos-discovery/blob/87f8cf8aa5229864360a9a4333f19915b303f02f/minos/api_gateway/discovery/domain/microservice.py#L82-L101

The set_data call catches exceptions raised from the database's client, and "transform" them into a True or False response. As the Micorservice.save method does processes those return values, the errors are silenced.

https://github.com/minos-framework/minos-discovery/blob/87f8cf8aa5229864360a9a4333f19915b303f02f/minos/api_gateway/discovery/database/client.py#L75-L82

I would bet that this bug is related with a small redis pool size, as it only support 10 simultaneous calls. This issue (https://github.com/minos-framework/minos-discovery/issues/70) is intended to increase that value.

https://github.com/minos-framework/minos-discovery/blob/87f8cf8aa5229864360a9a4333f19915b303f02f/minos/api_gateway/discovery/database/client.py#L48