filipecosta90 commented 3 years ago

Context

This PR kicks-off the RedisTimeSeries support for the tsbs benchmark suite.

RedisTimeSeries is a Redis Module adding a Time Series data structure to Redis.
The tsbs benchmark suite currently has two use cases, DevOps and IoT. This supplemental guide focus on RedisTimeSeries DevOps support.

Try it out

Spinning local RedisTimeSeries

Bellow you can find how to easily try the full RedisTimeSeries tsbs cycle prior merging using our RedisTimeSeries/tsbs fork:

How to build the required Go programs
How to generate the required datasets and queries
How to load the data into RedisTimeSeries
How to benchmark RedisTimeSeries query performance

To test locally you can use docker:

docker run -p 6379:6379 redis/redis-stack-server:edge

Installation of Go binaries using our fork

The easiest way to get and install the Go programs is to:

# Fetch TSBS and its dependencies
git clone https://github.com/RedisTimeSeries/tsbs --branch redistimeseries-devops
cd tsbs
make

Full cycle TSBS RedisTimeSeries scripts

Instead of calling tsbs redistimeseries binaries directly, we also supply scripts/*.sh for convenience with many of the flags set to a reasonable default for RedisTimeSeries database.

So for a Full cycle TSBS RedisTimeSeries benchmark, ensure that RedisTimeSeries is running and then use:

./scripts/full_cycle_minitest/full_cycle_minitest_redistimeseries.sh

codecov-commenter commented 3 years ago

Codecov Report

Merging #162 (a4efe90) into master (a7763a8) will decrease coverage by 1.65%. The diff coverage is 33.13%.

@@            Coverage Diff             @@
##           master     #162      +/-   ##
==========================================
- Coverage   65.25%   63.59%   -1.66%     
==========================================
  Files         145      149       +4     
  Lines        5966     6280     +314     
==========================================
+ Hits         3893     3994     +101     
- Misses       1960     2170     +210     
- Partials      113      116       +3

Impacted Files	Coverage Δ
cmd/tsbs_generate_queries/uses/devops/common.go	`87.50% <0.00%> (-5.84%)`	:arrow_down:
internal/inputs/generator_queries.go	`82.56% <ø> (ø)`
load/loader.go	`42.95% <0.00%> (-8.25%)`	:arrow_down:
pkg/data/usecases/common/simulator.go	`84.50% <0.00%> (-7.81%)`	:arrow_down:
pkg/data/usecases/devops/generate_data.go	`67.50% <0.00%> (-1.74%)`	:arrow_down:
...a/usecases/devops/generic_metrics_generate_data.go	`85.10% <0.00%> (-1.86%)`	:arrow_down:
pkg/data/usecases/iot/simulator.go	`86.48% <0.00%> (-0.79%)`	:arrow_down:
pkg/query/benchmarker.go	`61.00% <0.00%> (-12.50%)`	:arrow_down:
pkg/query/redistimeseries.go	`0.00% <0.00%> (ø)`
pkg/query/stat_processor.go	`7.03% <0.00%> (-3.20%)`	:arrow_down:
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a7763a8...a4efe90. Read the comment docs.

CLAassistant commented 2 years ago

All committers have signed the CLA.

seybi87 commented 1 year ago

Hi @filipecosta90 , great work on extending TSBS for Redis.

We have tried to run your extensions against the latest Redis OSS container but encountered the following error during the insert phase:

./tsbs_generate_queries --queries 2000 --query-type single-groupby-1-1-1 --scale 1 --seed 123 --timestamp-start 2022-01-01T00:00:00Z --timestamp-end 2022-01-02T00:00:00Z --use-case cpu-only --format redistimeseries --file /tmp/queries

./tsbs_generate_data --initial-scale 1 --scale 10 --seed 123 --timestamp-start 2022-01-01T00:00:00Z --timestamp-end 2022-01-02T00:00:00Z --use-case cpu-only --format redistimeseries --file /tmp/data

./tsbs_load_redistimeseries --host 172.17.0.2:6379 --batch-size 1000 --insert-intervals 1,2 --seed 123 --workers 2 --file /tmp/data
2022/09/16 16:21:19 Starting benchmark
2022/09/16 16:21:19 Using compression:  compressed
2022/09/16 16:21:19 creating DS from /tmp/data
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
2022/09/16 16:21:19 Flush failed with failed to decode pipeline CmdAction 'TS.CREATE' with keys [{host_0}_cpu_usage_user]: ERR unknown command 'TS.CREATE', with args beginning with: '{host_0}_cpu_usage_user' 'compressed' 'LABELS' 'hostname' 'host_0' 'region' 'eu-central-1' 'datacenter' 'eu-central-1a' 'rack'

Any idea what causes the problem?

filipecosta90 commented 1 year ago

Hi @filipecosta90 , great work on extending TSBS for Redis.

We have tried to run your extensions against the latest Redis OSS container but encountered the following error during the insert phase:

./tsbs_generate_queries --queries 2000 --query-type single-groupby-1-1-1 --scale 1 --seed 123 --timestamp-start 2022-01-01T00:00:00Z --timestamp-end 2022-01-02T00:00:00Z --use-case cpu-only --format redistimeseries --file /tmp/queries

./tsbs_generate_data --initial-scale 1 --scale 10 --seed 123 --timestamp-start 2022-01-01T00:00:00Z --timestamp-end 2022-01-02T00:00:00Z --use-case cpu-only --format redistimeseries --file /tmp/data

./tsbs_load_redistimeseries --host 172.17.0.2:6379 --batch-size 1000 --insert-intervals 1,2 --seed 123 --workers 2 --file /tmp/data
2022/09/16 16:21:19 Starting benchmark
2022/09/16 16:21:19 Using compression:  compressed
2022/09/16 16:21:19 creating DS from /tmp/data
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
2022/09/16 16:21:19 Flush failed with failed to decode pipeline CmdAction 'TS.CREATE' with keys [{host_0}_cpu_usage_user]: ERR unknown command 'TS.CREATE', with args beginning with: '{host_0}_cpu_usage_user' 'compressed' 'LABELS' 'hostname' 'host_0' 'region' 'eu-central-1' 'datacenter' 'eu-central-1a' 'rack'

Any idea what causes the problem?

Hi there @seybi87, given RedisTimeseries is an addition (module) on top of the vanilla Redis it's not present as part of the Redis OSS container. You can try it out via:

docker run -p 6379:6379 redis/redis-stack-server:edge

which contains all the latest versions of the Modules supported/developed by Redis Ltd.

seybi87 commented 1 year ago

@filipecosta90 thanks a lot for the explanation, this resolved the issue.

bamaas commented 7 months ago

Any updates on this?

jonatas commented 7 months ago

Hi @bamaa, I'll check internally if anyone can take over but I suggest just use the branch from the PR while it's not merged.

timescale / tsbs

RedisTimeSeries DevOps use-case initial support #162