philjb
commented
3 weeks ago I'm not sure what you mean by aggregate (could you explain?) but I believe these docs address what I think is happening to you:
https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_tutorial/#duplicate-points
(please close this ticket if so)
I am pushing data from NATS to InfluxDB using Telegraf. The data is being generated continuously, which results in multiple entries having the same timestamp. My timestamps are in nanoseconds, but in InfluxDB, the data is being aggregated, causing a loss of data points in my Grafana dashboard.
Can any one suggest how to avoid data loss.
example data.
requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241015-1415,samplerType=transaction,subject=nats.subject.requestsraw,testName=Test time=1728981942717004500,count=1,errorCount=0,sentBytes=1467,timeStamp=1728981940732,receivedBytes=59275,latency=0,processingTime=0,connectTime=0,responseTime=3151 1728981942717004544 requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241015-1415,samplerType=transaction,subject=nats.subject.requestsraw,testName=Test latency=0,receivedBytes=59275,time=1728981942717004500,errorCount=0,responseTime=1276,count=1,processingTime=0,sentBytes=1467,timeStamp=1728981942619,connectTime=0 1728981942717004544
Below is my telegraf configuration
[[inputs.nats_consumer]] servers = [“nats://localhost:4222”] subjects = [“nats.subject.requestsraw”] data_format = “xpath_json” xpath_native_types = true fieldexclude = [“subjects”, “runId”, “testName”, “nodeName”, “ObjectType”, “errorMessage”, “errorResponseBody”, “requestName”, “responseCode”, “result”, “samplerType”]
[[inputs.nats_consumer.xpath]] metric_name = “‘requestsRaw’” metric_selection = “/" timestamp = “time” timestamp_format = “unix_ns” field_selection = "” [inputs.nats_consumer.xpath.tags] runId = “string(/runId)” testName = “string(/testName)” nodeName = “string(/nodeName)” ObjectType = “string(/ObjectType)” errorMessage = “string(/errorMessage)” errorResponseBody = “string(/errorResponseBody)” requestName = “string(/requestName)” responseCode = “string(/responseCode)” result = “string(/result)” samplerType = “string(/samplerType)”