numaproj / numalogic-prometheus

AIOps for metrics in Prometheus
Apache License 2.0
31 stars 10 forks source link

chore: Upgrade to Numaflow 0.11 and Fix: Trainer Pod getting the slave redis client connection #173

Closed shashank10456 closed 8 months ago

shashank10456 commented 8 months ago

Fix for the issue:

Root cause of why the trainer pod is getting the slave redis client connection. The root cause for this is the way that we are importing the modules(preprocess, inference etc). When we are importing modules(Ex. inference), we are executing the global variable in inference module REDIS_CLIENT which in turn calculates global variable SENTINEL_CLIENT in Sentinel.py. There is a catch here. In python, the global variables are module scoped, because of this, when we are trying to calculate get_redis_client_from_conf in trainer.py module it is giving you the value of SENTINEL_CLIENT which it already calculated while importing inference module. Because of this, no matter what params you pass in while calling the get_redis_client_from_conf method, the client is returning the value it already has.

codecov-commenter commented 8 months ago

Codecov Report

Attention: 5 lines in your changes are missing coverage. Please review.

Comparison is base (e8c600b) 85.96% compared to head (cc6cd5d) 85.61%.

Files Patch % Lines
numaprom/clients/sentinel.py 16.66% 4 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #173 +/- ## ========================================== - Coverage 85.96% 85.61% -0.35% ========================================== Files 20 20 Lines 1154 1161 +7 Branches 118 119 +1 ========================================== + Hits 992 994 +2 - Misses 138 142 +4 - Partials 24 25 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.