jenkinsci / github-autostatus-plugin

Jenkins plugin to provide automatic status for multibranch jobs
https://plugins.jenkins.io/github-autostatus/
MIT License
52 stars 34 forks source link

Send results to InfluxDB in batches #94

Closed RickyGrassmuck closed 2 years ago

RickyGrassmuck commented 4 years ago

Currently, each datapoint generated by the plugin is being sent in it's own individual HTTP request to the InfluxDB API.

While this isn't an issue when the number of datapoints being sent are relatively low, this can lead to significant traffic and unnecessary write activity on the InfluxDB server when the number of datapoints increases.

The official recommendation from InfluxDB's documentation states that in most cases, each POST request to the server contains between 5,000-10,000 data points each.

https://docs.influxdata.com/influxdb/v1.7/concepts/glossary/#batch

Looking at the API docs for the request-body of an InfluxDB POST request, you can batch the individual datapoints by simply separating them using a newline.

https://docs.influxdata.com/influxdb/v1.7/tools/api/#request-body-1

I'm not a java developer but after looking through the codebase I believe the bulk of the requests end up coming from the notifyTestSuite() function when it loops over each test case in a test suite and calls notifyTestCase

        for (TestCase testCase : testSuite.getTestCases()) {
            notifyTestCase(jobName, suiteName, testCase, run);
        }

I opened PR #93 with what I think is a possible solution to this but I would highly recommend scrutinizing the work as I am very unfamiliar with the intricacies of Java!