eclipse / kapua

Eclipse Public License 2.0
222 stars 160 forks source link

:zap: [Datastore] Reduce the number of queries to keep the registry info updates #4081

Closed dseurotech closed 1 month ago

dseurotech commented 1 month ago

This pr adds a configuration switch that allows to skip a costly (and potentially unneccesary) fetch from elasticsearch currently performed for each metric not present in the cache (very inefficient in case of device messages with a lot of metrics). The behaviour can me configured by the following parameter: datastore.cache.metrics.fetchBeforeUpsert=true omit the configuration or leave at 'true' to maintain the current behaviour, set to 'false' to skip the fetch

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 87.50000% with 1 line in your changes missing coverage. Please review.

Project coverage is 52.30%. Comparing base (f15c972) to head (997258e). Report is 11 commits behind head on release-1.7.x.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/eclipse/kapua/pull/4081/graphs/tree.svg?width=650&height=150&src=pr&token=1P4N4CApH8&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse)](https://app.codecov.io/gh/eclipse/kapua/pull/4081?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse) ```diff @@ Coverage Diff @@ ## release-1.7.x #4081 +/- ## =================================================== - Coverage 52.34% 52.30% -0.05% Complexity 1864 1864 =================================================== Files 1633 1633 Lines 46163 46166 +3 Branches 4233 4234 +1 =================================================== - Hits 24165 24148 -17 - Misses 20604 20623 +19 - Partials 1394 1395 +1 ``` | [Files](https://app.codecov.io/gh/eclipse/kapua/pull/4081?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse) | Coverage Δ | | |---|---|---| | [...tastore/internal/setting/DatastoreSettingsKey.java](https://app.codecov.io/gh/eclipse/kapua/pull/4081?src=pr&el=tree&filepath=service%2Fdatastore%2Finternal%2Fsrc%2Fmain%2Fjava%2Forg%2Feclipse%2Fkapua%2Fservice%2Fdatastore%2Finternal%2Fsetting%2FDatastoreSettingsKey.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse#diff-c2VydmljZS9kYXRhc3RvcmUvaW50ZXJuYWwvc3JjL21haW4vamF2YS9vcmcvZWNsaXBzZS9rYXB1YS9zZXJ2aWNlL2RhdGFzdG9yZS9pbnRlcm5hbC9zZXR0aW5nL0RhdGFzdG9yZVNldHRpbmdzS2V5LmphdmE=) | `100.00% <100.00%> (ø)` | | | [...e/datastore/internal/MetricInfoRegistryFacade.java](https://app.codecov.io/gh/eclipse/kapua/pull/4081?src=pr&el=tree&filepath=service%2Fdatastore%2Finternal%2Fsrc%2Fmain%2Fjava%2Forg%2Feclipse%2Fkapua%2Fservice%2Fdatastore%2Finternal%2FMetricInfoRegistryFacade.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse#diff-c2VydmljZS9kYXRhc3RvcmUvaW50ZXJuYWwvc3JjL21haW4vamF2YS9vcmcvZWNsaXBzZS9rYXB1YS9zZXJ2aWNlL2RhdGFzdG9yZS9pbnRlcm5hbC9NZXRyaWNJbmZvUmVnaXN0cnlGYWNhZGUuamF2YQ==) | `65.42% <85.71%> (-0.30%)` | :arrow_down: | ... and [11 files with indirect coverage changes](https://app.codecov.io/gh/eclipse/kapua/pull/4081/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse)