search

dpattmann / prometheus-timestream-adapter

Use AWS Timestream as a remote storage database for Prometheus
Apache License 2.0
36 stars 12 forks source link

A record already exists with the same time #87

Closed ratovarius closed 2 years ago

ratovarius commented 2 years ago

After configuring remote write, I see the initial entries on the timestream db. After a while, I don't see new entries. Checking the logs with, k logs prometheus-prometheus-0 -n prometheus -c timestream-adapter I get,

{
  "level":"warn",
  "ts":1643716808.3651953,
  "caller":"prometheus-timestream-adapter/timestream.go:134",
  "msg":"Error sending samples to remote storage",
  "err":"RejectedRecordsException: One or more records have been rejected. See RejectedRecords for details.
{
  RespMetadata: {
    StatusCode: 419,
    RequestID: "FCP7ZH7MFKNMBE2FQWRVVVPAAI"
  },
  Message_: "One or more records have been rejected. See RejectedRecords for details.",
  RejectedRecords: [
    {
      Reason: "A record already exists with the same time, dimensions, measure name, and record version. A higher record version must be specified in order to update the measure value. Specifying record version is supported by the latest SDKs.",
      RecordIndex: 85
    },
    ...
  ]
}",
  "storage":"prometheus-timestream-adapter","num_samples":100
}

Am I configuring something wrong on the prometheus side?

dpattmann commented 2 years ago

Hi @ratovarius, this errors occurs when when you try to send the exact same metric multiple times.