Support native --out influxdb=filename.log InfluxDB batch line format logs

[centminmod]

Feature Description

Background

As discussed on the k6 community https://community.k6.io/t/k6-testing-with-influxdb-grafana-prometheus/4857. When using --out to InfluxDB for high VUs concurrency tests, InfluxDB may not be able to keep up with the amount of data set from k6. Then we'd need to tweak the K6_INFLUXDB_PUSH_INTERVAL and K6_INFLUXDB_CONCURRENT_WRITES and strip some metrics/tags to reduce the data set size (discussed here).

During my research one suggestion was to run k6 with --out to a JSON file and parse it for manual insertion into InfluxDB.

Feature Request

Maybe k6 can add a --out influxdb=filename.log option to write the saved output to filename.log in native InfluxDB batch line format for easier insertion later?

I tried my hand at parsing the k6 JSON logs and got them in InfluxDB 1.8 batch line format at https://github.com/centminmod/k6-benchmarking/blob/master/logs/k6-log-parsing.md, but if it can be done natively in k6, then it would save a lot of parsing time on large tests.

[na--]

:thinking: This sounds like a nice idea. The only problem is that the current influxdb output is in a bit of a maintenance mode. The k6 InfluxDB output still supports only InfluxDB v1.x because the v2.x SDK was super bloated and increased the k6 binary size by a lot (and had some other issues). For more details: https://github.com/grafana/k6/issues/1730

That said, I am not sure the InfluxDB v1.x and v2.x line protocols are different :thinking: They seem similar enough at a glance:

• https://docs.influxdata.com/influxdb/v1.8/write_protocols/line_protocol_tutorial/
• https://docs.influxdata.com/influxdb/v2.4/reference/syntax/line-protocol/

So it might be a good idea to implement this even in the current influxdb output? :thinking:

[centminmod]

Ah the line protocols do look similar which would make this requested feature nice to have :)

I still new to InfluxDB so using 1.8 so have no clues on 2.x.