perf: optimize RecordBatch to HttpOutput conversion

GreptimeTeam / greptimedb

An open-source, cloud-native, unified time series database for metrics, logs and events with SQL/PromQL supported. Available on GreptimeCloud.

Apache License 2.0

4.23k stars 303 forks source link

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

Optimize HttpRecordsOutput::try_new() method. Reduce the consumption of converting 409600 rows from ~400ms to ~120ms. Saves ~70% CPU.

This patch also adds a microbenchmark for that method.

Key optimizations are:

Change the algorithm to do column-row conversion
Change the underlying type of Value::String from Bytes to String
Avoid all unnecessary Clones

Now the entire conversion method only has one clone that takes data from the data vector, and one write that writes the serialized content to the result vector.

Checklist

[x] I have written the necessary rustdoc comments.
[x] I have added the necessary unit tests and integration tests.
[ ] This PR requires documentation updates.

Codecov Report

Attention: Patch coverage is 60.00000% with 22 lines in your changes missing coverage. Please review.

Project coverage is 84.79%. Comparing base (cc2f7ef) to head (4923089). Report is 5 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #4178 +/- ## ========================================== - Coverage 85.12% 84.79% -0.33% ========================================== Files 1020 1022 +2 Lines 179635 179887 +252 ========================================== - Hits 152920 152543 -377 - Misses 26715 27344 +629 ```

GreptimeTeam / greptimedb