Unexpected behavior with empty data of one of server group

[klimenk0aa]

Hi I have a promxy instance with two server_groups. I use victoriametrics as my target. When there are data gaps in one server group, promxy returns the result without taking that group into account. Details in the screenshots below. raw data from first group

raw data from second group

and promxy result, is very similar to the result of the second group

but if you shift the query period so that the failure in the first group is not included in the result, the result in promxy is very different

Whether this is a bug or normal behavior? thx

[jacksontj]

This will depend a lot of the query and your configuration. Generally speaking promxy will merge timeseries (fill holes) for matching series. This can be somewhat complex depending on your labeling configuration for a server group. I'm addition, the "holes" are determined by the response from downstream. So sometimes is the lookback is long on the downstream it will interpolate some value instead of having a hole.

To debug further I'd need the config file and the trace log output (which shows the queries and results from downstream)

[klimenk0aa]

Add some files config config.yaml.txt

log log.txt

downstream exp-victoriam01 node1.json

downstream exp-victoriam03 node3.json

[jacksontj]

My apologies for the long response here -- but (un)fortunately this seems to be another instance of https://github.com/jacksontj/promxy/issues/260. Basically from looking at your configuration there is no label differentiation between the servergroups so when the merge occurs both seem "the same". The simplest way to fix this is to add a label at the servergroup level using labels. If you don't want that label to be propagated outside of promxy then you can use a __ prefixed label.

Hopefully that resolves the issue for you!

[klimenk0aa]

thanks a lot