Civil
commented
3 years ago Can you describe a way how can I replicate that behavior on an artificial setup (e.x. imagine that I've just installed VM and carbonapi, what should I feed it to trigger that behavior)?
As my test setup for VM + carbonapi looks like that:
write path: telegraf -> (influxdb writer) -> VM
read path: carbonapi -> VM
With that setup I've never seen such behavior. I'll try to reproduce it though again (not sure how I can delay data merges, it might be dependent on what VM returns). But I guess in my setup I do not usually see unmerged data (I do not have that much of a load, I'll try to throttle disk i/o for vm though to replicate that behavior).
It might be helpful if you can provide what VM returns on typical request from carbonapi (any metric that would trigger that behavior would work). You can enable debug logs in carbonapi and it should eventually print exact query that it do towards VictoriaMetrics and then you can query some data and replace actual metric names with something that doesn't contain any sensitive information (as well as for example replacing all data with 1.0 instead of whatever value there was)
keliss
Describe the bug Hi. Any metric queried from carbonapi results in multiple absolutely identical series with the same name. The number of duplicates seems to depend on whether there were any data merges in VictoriaMetrics (e.g. the number of duplicates can be 5, and after forcing a merge it reduces to 2).
CarbonAPI Version carbonapi-0.15.4~1-1
Logs Literally any metric and request should reproduce the issue, but let me know if request-response logs are still needed.
CarbonAPI Configuration:
VictoriaMetrics Configuration: Cluster version, 1.64.1. Metrics are ingested like this: carbon-c-relay -> vmagent -> vminsert -> vmstorage, and read like this: grafana -> carbonapi -> vmselect -> vmstorage.