Ability to add additional options into a writeRequest

[bcorbett6]

Use Case

For customers who need to tailor their write client based on the below config:

• batchSize
• flushInterval
• bufferLimit
• jitterInterval
• retryInterval

This is useful for developing a productionised and durable write-client

Expected behavior

The ability to use the specified config options to tailor how data is sent to InfluxDB

Actual behavior

I don't believe these write options are yet included

Additional info

No response

[bednar]

@bcorbett6, thank you for your proposal. We are currently prioritizing adding support for the web environment. Once that is accomplished, we can then focus on implementing support for batching writes.

[guoxiangke]

batching writes needed, thanks.

[bednar]

Hi @guoxiangke,

Thank you for your interest in our client.

Regarding batching writes, what is your preferred approach? Would you like a background service that writes data asynchronously on a separate thread? Or would you prefer a straightforward writer that divides data into batches, writes asynchronously, and then returns the result?

Regards

[anfreita]

@bednar

For lack of response to your earlier questions, I think some simple retry logic in the existing node implementation would be fantastic. Something to mitigate for potential 413s and 429s when loading larger data sets.

You maybe have more insight into this than I, but I think that ideally it might be nice to maintain parity with the telegraf influx plugin here: https://github.com/influxdata/telegraf/blob/master/plugins/outputs/influxdb_v2/http.go

[bednar]

Hi @anfreita,

Thank you for your feedback and suggestions.

I wanted to update you that we have added the following enhancements to our development backlog for the v3 clients:

1. Proper handling of the HTTP 429 status (Too Many Requests) - We will implement a retry mechanism that intelligently manages rate limiting issues by retrying requests after an appropriate delay.

2. Proper handling of the HTTP 413 status (Payload Too Large) - We plan to enhance the client to automatically split requests that exceed the server's size limits, ensuring that large payloads are handled more efficiently.

These enhancements are aimed at improving the robustness and usability of our clients. We'll be looking into these as part of our next set of enhancements.

Your input is invaluable, and we appreciate your contribution to making our tools better. We will keep you updated on our progress and when you can expect these features to be implemented.

Best regards.