logstash-plugins / logstash-output-influxdb

Apache License 2.0
58 stars 79 forks source link

Fix data corruption when creating batches. #11

Open Wilfred opened 9 years ago

Wilfred commented 9 years ago

Previously, we were appending events to the event collection even when the columns did not match!

Given the following three events:

{
  "name": "my-series",
  "columns": [
    "cpu.user",
    "time"
  ],
  "points": [
    [
      5.0,
      1426288467
    ]
  ]
}
{
  "name": "my-series",
  "columns": [
    "memory.used",
    "time"
  ],
  "points": [
    [
      1000,
      1426288467
    ]
  ]
}
{
  "name": "my-series",
  "columns": [
    "cpu.user",
    "time"
  ],
  "points": [
    [
      6.0,
      1426288477
    ]
  ]
}

We would end up with the following batch:

[
  {
    "name": "my-series",
    "columns": [
      "cpu.user",
      "time"
    ],
    "points": [
      [
        5.0,
        1426288467
      ],
      [
        6.0,
        1426288467
      ]
    ]
  },
  {
    "name": "my-series",
    "columns": [
      "memory.used",
      "time"
    ],
    "points": [
      [
        1000.0,
        1426288467
      ],
      [
        6.0, # this shouldn't be here!
        1426288467
      ]
    ]
  }
]
brupm commented 9 years ago

I am not sure I understand this one. @Wilfred by columns do you mean inside the datapoints?

Wilfred commented 9 years ago

Yes, inside the datapoints we're taking values from the wrong event. Does that make sense?

brupm commented 9 years ago

Yes it does.

elasticsearch-release commented 8 years ago

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.