search

cmu-db / benchbase

Multi-DBMS SQL Benchmarking Framework via JDBC
https://db.cs.cmu.edu/projects/benchbase/
Other
471 stars 185 forks source link

YCSB table creation never ends on Cloud Spanner #91

Closed stonewhitener closed 2 years ago

stonewhitener commented 2 years ago

I am trying to benchmark Cloud Spanner with YCSB, however, YCSB table creation never ends.

$ java -jar benchbase.jar -b ycsb -c config/spanner/ycsb_config.xml --create=true --load=true --execute=true
[DEBUG] 2021-12-17 05:56:26,426 [main]  com.oltpbenchmark.api.StatementDialects getSQLDialectPath - No dialect file in benchmarks/ycsb/dialect-spanner.xml
[DEBUG] 2021-12-17 05:56:26,428 [main]  com.oltpbenchmark.api.StatementDialects load - SKIP - No SQL dialect file was given.
[INFO ] 2021-12-17 05:56:26,440 [main]  com.oltpbenchmark.DBWorkload main - ======================================================================

Benchmark:     YCSB {com.oltpbenchmark.benchmarks.ycsb.YCSBBenchmark}
Configuration: config/spanner/ycsb_config.xml
Type:          SPANNER
Driver:        com.google.cloud.spanner.jdbc.JdbcDriver
URL:           jdbc:cloudspanner:/projects/foo/instances/bar/databases/sira?credentials=/Users/sira/Code/benchbase/config/spanner/sira-personal-project-48ff2575be0b.json
Isolation:     TRANSACTION_SERIALIZABLE
Batch Size:    128
Scale Factor:  1.0
Terminals:     1

[INFO ] 2021-12-17 05:56:26,440 [main]  com.oltpbenchmark.DBWorkload main - ======================================================================
[DEBUG] 2021-12-17 05:56:26,464 [main]  com.oltpbenchmark.DBWorkload main - Using the following transaction types: [com.oltpbenchmark.api.TransactionType$Invalid/00, com.oltpbenchmark.benchmarks.ycsb.procedures.ReadRecord/01, com.oltpbenchmark.benchmarks.ycsb.procedures.InsertRecord/02, com.oltpbenchmark.benchmarks.ycsb.procedures.ScanRecord/03, com.oltpbenchmark.benchmarks.ycsb.procedures.UpdateRecord/04, com.oltpbenchmark.benchmarks.ycsb.procedures.DeleteRecord/05, com.oltpbenchmark.benchmarks.ycsb.procedures.ReadModifyWriteRecord/06]
[DEBUG] 2021-12-17 05:56:26,465 [main]  com.oltpbenchmark.DBWorkload main - Num groupings: 0
[DEBUG] 2021-12-17 05:56:26,474 [main]  com.oltpbenchmark.DBWorkload isBooleanOptionSet - CommandLine has option 'create'. Checking whether set to true
[DEBUG] 2021-12-17 05:56:26,474 [main]  com.oltpbenchmark.DBWorkload isBooleanOptionSet - CommandLine create => true
[INFO ] 2021-12-17 05:56:26,474 [main]  com.oltpbenchmark.DBWorkload main - Creating new YCSB database...
[DEBUG] 2021-12-17 05:56:26,474 [main]  com.oltpbenchmark.DBWorkload runCreator - Creating ycsb Database
[DEBUG] 2021-12-17 05:56:27,802 [main]  com.oltpbenchmark.api.BenchmarkModule createDatabase - Executing script [benchmarks/ycsb/ddl-spanner.sql] for database type [SPANNER]
[DEBUG] 2021-12-17 05:56:27,802 [main]  com.oltpbenchmark.util.ScriptRunner runScript - trying to find file by path benchmarks/ycsb/ddl-spanner.sql
[DEBUG] 2021-12-17 05:56:27,804 [main]  com.oltpbenchmark.util.ScriptRunner runScript - CREATE TABLE usertable (
[DEBUG] 2021-12-17 05:56:27,804 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     ycsb_key INT64,
[DEBUG] 2021-12-17 05:56:27,804 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field1   STRING(100),
[DEBUG] 2021-12-17 05:56:27,804 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field2   STRING(100),
[DEBUG] 2021-12-17 05:56:27,805 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field3   STRING(100),
[DEBUG] 2021-12-17 05:56:27,805 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field4   STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field5   STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field6   STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field7   STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field8   STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field9   STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript -     field10  STRING(100),
[DEBUG] 2021-12-17 05:56:27,806 [main]  com.oltpbenchmark.util.ScriptRunner runScript - ) PRIMARY KEY (ycsb_key);

(never ends...)

I have already checked that the credentials work well and the same table can be created using simple code like this.

I couldn't make any progress on this problem though I investigated the source code.

timveil commented 2 years ago

@y-sira I was not able to recreate your issue. I just tested with the Spanner Emulator and was able to successfully run ycsb against the emulator following the steps here... https://github.com/cmu-db/benchbase/tree/main/docker/spanner-emulator

Have you had a chance to try that?

timveil commented 2 years ago

One thing i am noticing is that there may be a problem with the latest version of the spanner jdbc driver. We recently merged in the latest version 2.5.5. I'm running into problems testing against the emulator with this version. If i revert the jdbc driver back to version 2.5.1 ycsb executes fine. If you are running the latest code from the main branch you may want to change the pom.xml file to point to while more research is done.

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-spanner-jdbc</artifactId>
    <version>2.5.1</version>
</dependency>
stonewhitener commented 2 years ago

If i revert the jdbc driver back to version 2.5.1 ycsb executes fine.

It works for me too.

I will share my experience on my real Spanner instance after further inspection. (This issue can be closed.)

timveil commented 2 years ago

this issue will ultimately be resolved when #97 is closed