Questdb benchmark support

[patrickSpaceSurfer]

This pull request adds support for QuestDB, an open source time series database. QuestDB supports ingestion over InfluxDB line protocol, making tsbs_generate_data compatible with the influx format, and tsbs_load_questdb very similar to tsbs_load_influx.

For queries, QuestDB supports SQL which is executed over an /exec HTTP endpoint on port 9000 by default, and all query types in the suite are supported using the questdb format for both iot and cpu-only (DevOps) use cases.

[CLAassistant]

All committers have signed the CLA.

[codecov-commenter]

Codecov Report

Merging #157 (264163f) into master (45b6321) will increase coverage by 0.45%. The diff coverage is 75.53%.

@@            Coverage Diff             @@
##           master     #157      +/-   ##
==========================================
+ Coverage   65.25%   65.71%   +0.45%     
==========================================
  Files         145      153       +8     
  Lines        5966     6244     +278     
==========================================
+ Hits         3893     4103     +210     
- Misses       1960     2020      +60     
- Partials      113      121       +8     

Impacted Files	Coverage Δ
cmd/tsbs_load_questdb/creator.go	0.00% <0.00%> (ø)
pkg/targets/questdb/implemented_target.go	0.00% <0.00%> (ø)
cmd/tsbs_load_questdb/main.go	56.25% <56.25%> (ø)
cmd/tsbs_load_questdb/process.go	66.66% <66.66%> (ø)
cmd/tsbs_load_questdb/scan.go	80.00% <80.00%> (ø)
pkg/targets/questdb/serializer.go	80.43% <80.43%> (ø)
.../tsbs_generate_queries/databases/questdb/common.go	88.88% <88.88%> (ø)
.../tsbs_generate_queries/databases/questdb/devops.go	100.00% <100.00%> (ø)
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 45b6321...264163f. Read the comment docs.

[jonatas]

@patrickSpaceSurfer just a heads up that increasing the size of the machine to 6 cores and 4 GB of RAM makes everything works as expected 🎉

Here is full_mini_cycle I created to test until it works:

FORMATS=influx TS_END=2016-01-01T02:00:00Z bash ./scripts/generate_data.sh
NUM_WORKERS=4
./scripts/load/load_questdb.sh
./scripts/run_queries/run_queries_questdb.sh

[patrickSpaceSurfer]

Hello @patrickSpaceSurfer! thanks for adding the questDB here!

Can you please rebase the PR at the top of the master branch and run make fmt or go fmt ./... to have the code formatted?

We also have a scripts/full_cycle_minitest/ in which you can also add one new bash script to run a full cycle of the scripts from questDB. It would be great and also guide the next users that will try to benchmark it ;)

The coverage also degraded with introducing new files and missing tests for it, can you please add the missing tests?

I have merged the latest master, reformatted the code and added a minitest script.

With regards to coverage, I did add some new tests a few days ago, but it looks like the coverage report has not run since then. How do I run it?

[jonatas]

@patrickSpaceSurfer you can run locally make coverage.

[patrickSpaceSurfer]

@patrickSpaceSurfer you can run locally make coverage.

I have added more tests, coverage is now similar to other databases

pim@patrick$ > make coverage | grep quest
ok      github.com/timescale/tsbs/cmd/tsbs_generate_queries/databases/questdb   0.057s  coverage: 98.5% of statements
ok      github.com/timescale/tsbs/cmd/tsbs_load_questdb 0.139s  coverage: 54.2% of statements
?       github.com/timescale/tsbs/cmd/tsbs_run_queries_questdb  [no test files]
ok      github.com/timescale/tsbs/pkg/targets/questdb   0.033s  coverage: 75.0% of statements

[bsmth]

Hi @jonatas, @RobAtticus thanks for the assistance with the PR so far. Is there anything else outstanding or is this ready to be merged? Thanks a lot! 🙌🏻