Open eivanov89 opened 1 year ago
Hello, I am Blathers. I am here to help you get the issue triaged.
Hoot - a bug! Though bugs are the bane of my existence, rest assured the wretched thing will get the best of care here.
I have CC'd a few people who may be able to assist you:
If we have not gotten back to your issue within a few business days, you can try the following:
:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.
I think the issue here is that --insert-count
only applies to cockroach workload init ycsb
and not cockroach workload run ycsb
.
It seems to be a weird difference in 99%:
./cockroach/cockroach workload run ycsb --workload c --request-distribution uniform --max-ops 5000000 --insert-count 1 --display-every 10s postgresql://root@ydb-vla-dev04-000.search.yandex.net:26257?sslmode=disable --concurrency 1
I230310 12:22:46.804731 1 workload/cli/run.go:427 [-] 1 creating load generator...
I230310 12:22:46.824254 1 workload/cli/run.go:458 [-] 2 creating load generator... done (took 19.522111ms)
_elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
10.0s 0 4.0 4.0 251.7 260.0 268.4 268.4 read
20.0s 0 4.1 4.0 251.7 260.0 268.4 268.4 read
30.0s 0 4.0 4.0 251.7 268.4 302.0 302.0 read
...
eivanov89@ydb-vla-dev02-000:~$ ./cockroach/cockroach workload run ycsb --workload c --request-distribution uniform --max-ops 5000000 --insert-count 1000 --display-every 10s postgresql://root@ydb-vla-dev04-000.search.yandex.net:26257?sslmode=disable --concurrency 1
I230310 12:25:55.670542 1 workload/cli/run.go:427 [-] 1 creating load generator...
I230310 12:25:55.690632 1 workload/cli/run.go:458 [-] 2 creating load generator... done (took 20.088996ms)
_elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
10.0s 0 321.1 321.8 2.6 2.9 3.3 268.4 read
20.0s 0 364.0 342.9 2.6 2.8 3.1 251.7 read
30.0s 0 385.4 357.1 2.6 2.8 3.1 8.9 read
If I remove --insert-count then result is similar to --insert-count 10000 (the default value of insert-count).
I think the issue here is that
--insert-count
only applies tocockroach workload init ycsb
and notcockroach workload run ycsb
.
But then shouldn't there be a validation and error in case of cockroach workload run ycsb
?
It does seem to also affect the workload. The workload does not validate that you've set it up correctly. The assumption of the workload, to the extent that I understand it, is that these flags are set to the same value between init
and run
. It looks like the flag both controls the initial data load and the number of rows available to the workload.
I think an ask for run
to validate that the correct amount of data exists is a fine ask, and PRs are welcome.
It looks like the flag both controls the initial data load and the number of rows available to the workload.
Though imho the flag "--record-count" seems to be better to specify the initial data size. According the help
--insert-count int Number of rows to sequentially insert before beginning workload. (default 10000)
--insert-start int Key to start initial sequential insertions from. (default 0)
--record-count int Key to start workload insertions from. Must be >= insert-start + insert-count. (Default: insert-start + insert-count)
So that I upload the data with combination of --insert-count and possibly --insert-start and then run workloads either by specifying same flags (a little bit confusing) or just --record-count.
It's very confusing that in the original YCSB same properties have different meaning:
Describe the problem
According the help it is a Number of rows to sequentially insert before beginning workload. I can assume that it either affects workloads which do insert operations or it forces some insertions before any workload.
Example command:
In cluster I have 8 servers, each one is 128 cores, 4 NVMe disks. There are 4 cockroach instances with 32 cores (set by taskset) and 1 disk per server. Replication factor is default.
Jira issue: CRDB-25230