brianfrankcooper / YCSB

Yahoo! Cloud Serving Benchmark
Apache License 2.0
4.88k stars 2.22k forks source link

Could not connect to Couchbase Bucket #1715

Open alexandvo opened 3 weeks ago

alexandvo commented 3 weeks ago

I'm trying to use the load command to load documents into my couchbase community server node that is deployed on my local machine. I am not using any containers or vms, I just set up the a single node inside a cluster on my local machine using Couchbase's Community Server installer. It is Couchbase Community Server version 7.6.1 on Windows. I have tested manually creating a document to my bucket on my cluster by using their Python SDK and it works fine. It successfully adds a document to the bucket.

My Config: I have a single bucket named: test I have a cluster named: cluster I have one server on this cluster running on my local machine I have an admin user with the following credentials: username: test password: password

The YCSB load command I used while in the ycsb-0.17.0 directory in my local machine: bin/ycsb load couchbase2 -s -P workloads/workloada -p couchbase.bucket=test -p couchbase.password=password -p recordcount=1000

I saw a previous issue talking about a similar problem where it was solved by making the admin account have the same name as the bucket to insert to, but I still have the issue even after doing this.

After executing this command it doesn't make any inserts to my cluster and it outputs that it did 0 operations...

Here is the full output with the errors after I execute this load command:

C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0>"C:\Program Files\Java\jdk1.8.0_321\bin\java.exe" -classpath "C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\conf;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\lib\core-0.17.0.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\lib\HdrHistogram-2.1.4.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\lib\htrace-core4-4.1.0-incubating.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\lib\jackson-core-asl-1.9.4.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\lib\jackson-mapper-asl-1.9.4.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\couchbase2-binding\lib\core-io-1.3.1.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\couchbase2-binding\lib\couchbase2-binding-0.17.0.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\couchbase2-binding\lib\java-client-2.3.1.jar;C:\Users\alext\Documents\Code\couchbase-testing\ycsb\ycsb-0.17.0\couchbase2-binding\lib\rxjava-1.1.5.jar" site.ycsb.Client -load -db site.ycsb.db.couchbase2.Couchbase2Client -s -P workloads/workloadb -p couchbase.password=password Command line: -load -db site.ycsb.db.couchbase2.Couchbase2Client -s -P workloads/workloadb -p couchbase.password=password YCSB Client 0.17.0

Loading workload... Starting test. 2024-07-11 16:24:27:526 0 sec: 0 operations; est completion in 0 second Jul 11, 2024 4:24:27 PM site.ycsb.db.couchbase2.Couchbase2Client logParams INFO: ===> Using Params: host=127.0.0.1, bucket=default, upsert=false, persistTo=NONE, replicateTo=NONE, syncMutResponse=true, adhoc=false, kv=true, maxParallelism=1, queryEndpoints=1, kvEndpoints=1, queryEndpoints=1, epoll=false, boost=3, networkMetricsInterval=0, runtimeMetricsInterval=0 Jul 11, 2024 4:24:27 PM com.couchbase.client.core.CouchbaseCore INFO: CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword=false, sslKeystore=null, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=20, computationPoolSize=20, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.3.1 (git: 2.3.1, core: 1.3.1), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=10000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=true, disconnectTimeout=25000, requestBufferWaitStrategy=com.couchbase.client.core.env.DefaultCoreEnvironment$2@407e6c, queryTimeout=75000, viewTimeout=75000, kvTimeout=10000, connectTimeout=30000, dnsSrvEnabled=false} 2024-07-11 16:24:37:528 10 sec: 0 operations; est completion in 106751991167300 days 15 hours 2024-07-11 16:24:47:530 20 sec: 0 operations; est completion in 106751991167300 days 15 hours 2024-07-11 16:24:57:520 30 sec: 0 operations; est completion in 106751991167300 days 15 hours site.ycsb.DBException: Could not connect to Couchbase Bucket. at site.ycsb.db.couchbase2.Couchbase2Client.init(Couchbase2Client.java:208) at site.ycsb.DBWrapper.init(DBWrapper.java:86) at site.ycsb.ClientThread.run(ClientThread.java:91) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.RuntimeException: java.util.concurrent.TimeoutException at com.couchbase.client.java.util.Blocking.blockForSingle(Blocking.java:71) at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:290) at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:271) at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:260) at site.ycsb.db.couchbase2.Couchbase2Client.init(Couchbase2Client.java:205) ... 3 more Caused by: java.util.concurrent.TimeoutException ... 8 more site.ycsb.DBException: Could not connect to Couchbase Bucket. at site.ycsb.db.couchbase2.Couchbase2Client.init(Couchbase2Client.java:208) at site.ycsb.DBWrapper.init(DBWrapper.java:86) at site.ycsb.ClientThread.run(ClientThread.java:91) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.RuntimeException: java.util.concurrent.TimeoutException at com.couchbase.client.java.util.Blocking.blockForSingle(Blocking.java:71) at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:290) at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:271) at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:260) at site.ycsb.db.couchbase2.Couchbase2Client.init(Couchbase2Client.java:205) ... 3 more Caused by: java.util.concurrent.TimeoutException ... 8 more 2024-07-11 16:24:57:874 30 sec: 0 operations; est completion in 106751991167300 days 15 hours [OVERALL], RunTime(ms), 30359 [OVERALL], Throughput(ops/sec), 0.0 [TOTAL_GCS_Copy], Count, 5 [TOTAL_GC_TIME_Copy], Time(ms), 8 [TOTAL_GCTIME%_Copy], Time(%), 0.026351329095161236 [TOTAL_GCS_MarkSweepCompact], Count, 0 [TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0 [TOTAL_GCTIME%_MarkSweepCompact], Time(%), 0.0 [TOTAL_GCs], Count, 5 [TOTAL_GC_TIME], Time(ms), 8 [TOTAL_GCTIME%], Time(%), 0.026351329095161236

seybi87 commented 3 weeks ago

Hi @alexandvo feel free to have a look at our YCSB result files which include the CLI parameters: https://github.com/benchANT/database-ranking/tree/main/CRUD-general-purpose/2022/ranking_batch-0_couchbase-ce-xsmall-small_0_ZoCTfpqe

They were run with the YCSB Client 0.17.0 using the Couchbase2 binding against Couchbase CE 7.0.0 and it should also work against CE 7.6

alexandvo commented 3 weeks ago

Hi @alexandvo feel free to have a look at our YCSB result files which include the CLI parameters: https://github.com/benchANT/database-ranking/tree/main/CRUD-general-purpose/2022/ranking_batch-0_couchbase-ce-xsmall-small_0_ZoCTfpqe

They were run with the YCSB Client 0.17.0 using the Couchbase2 binding against Couchbase CE 7.0.0 and it should also work against CE 7.6

Thank you for providing me with your YCSB result files. I recently get passed connecting to the node and the bucket I think since it now says: "INFO: Connected to Node 3.90.221.18 Jul 12, 2024 6:44:19 PM com.couchbase.client.core.config.DefaultConfigurationProvider$8 call INFO: Opened bucket couch" but it throws a new error saying "Request cancelled in-flight" and this occurs when I used a similar load command as the one y'all used for your benchmarking of couchbase in ec2 large. The only things I changed were bucket name, username, password, and host. This error also occurs with the basic load command as follows: bin/ycsb load couchbase2 -s -P workloads/workloada -p couchbase.host= -p couchbase.bucket=couch -p couchbase.user=couch -p couchbase.password=password. Did you ever run into this issue of request being cancelled in-flight? If so, how did you solve it? Here is the full error by the way: YCSB Client 0.17.0

Loading workload... Starting test. 2024-07-12 18:44:17:832 0 sec: 0 operations; est completion in 0 second WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.couchbase.client.deps.io.netty.util.internal.PlatformDependent0 (file:/home/ubuntu/ycsb-0.17.0/couchbase2-binding/lib/core-io-1.3.1.jar) to field java.nio.Buffer.address WARNING: Please consider reporting this to the maintainers of com.couchbase.client.deps.io.netty.util.internal.PlatformDependent0 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Jul 12, 2024 6:44:17 PM com.couchbase.client.deps.io.netty.util.internal.PlatformDependent INFO: Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system unstability. Jul 12, 2024 6:44:18 PM com.couchbase.client.core.env.DefaultCoreEnvironment INFO: ioPoolSize is less than 3 (1), setting to: 3 Jul 12, 2024 6:44:18 PM com.couchbase.client.core.env.DefaultCoreEnvironment INFO: computationPoolSize is less than 3 (1), setting to: 3 Jul 12, 2024 6:44:18 PM site.ycsb.db.couchbase2.Couchbase2Client logParams INFO: ===> Using Params: host=3.90.221.18, bucket=couch, upsert=false, persistTo=NONE, replicateTo=NONE, syncMutResponse=true, adhoc=false, kv=true, maxParallelism=1, queryEndpoints=1, kvEndpoints=1, queryEndpoints=1, epoll=false, boost=3, networkMetricsInterval=0, runtimeMetricsInterval=0 Jul 12, 2024 6:44:18 PM com.couchbase.client.core.CouchbaseCore INFO: CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword=false, sslKeystore=null, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=3, computationPoolSize=3, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.3.1 (git: 2.3.1, core: 1.3.1), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=10000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=true, disconnectTimeout=25000, requestBufferWaitStrategy=com.couchbase.client.core.env.DefaultCoreEnvironment$2@67e7535b, queryTimeout=75000, viewTimeout=75000, kvTimeout=10000, connectTimeout=30000, dnsSrvEnabled=false} Jul 12, 2024 6:44:19 PM com.couchbase.client.core.node.CouchbaseNode signalConnected INFO: Connected to Node 3.90.221.18 Jul 12, 2024 6:44:19 PM com.couchbase.client.core.config.DefaultConfigurationProvider$8 call INFO: Opened bucket couch DBWrapper: report latency for each error is false and specific error codes to track for latency are: [] com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight. at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:486) at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:467) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:640) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:913) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:880) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:872) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:856) at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:695) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:374) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) at java.base/java.lang.Thread.run(Thread.java:829) Error inserting, not retrying any more. number of attempts: 1Insertion Retry Limit: 0 2024-07-12 18:44:19:942 2 sec: 0 operations; est completion in 106751991167300 days 15 hours [CLEANUP: Count=1, Max=1, Min=1, Avg=1, 90=1, 99=1, 99.9=1, 99.99=1] [INSERT: Count=0, Max=0, Min=9223372036854775807, Avg=NaN, 90=0, 99=0, 99.9=0, 99.99=0] [INSERT-FAILED: Count=1, Max=72255, Min=72192, Avg=72224, 90=72255, 99=72255, 99.9=72255, 99.99=72255] [OVERALL], RunTime(ms), 2229 [OVERALL], Throughput(ops/sec), 0.0 [TOTAL_GCS_Copy], Count, 10 [TOTAL_GC_TIME_Copy], Time(ms), 30 [TOTAL_GCTIME%_Copy], Time(%), 1.3458950201884252 [TOTAL_GCS_MarkSweepCompact], Count, 2 [TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 23 [TOTAL_GCTIME%_MarkSweepCompact], Time(%), 1.031852848811126 [TOTAL_GCs], Count, 12 [TOTAL_GC_TIME], Time(ms), 53 [TOTAL_GCTIME%], Time(%), 2.3777478689995513 [CLEANUP], Operations, 1 [CLEANUP], AverageLatency(us), 1.0 [CLEANUP], MinLatency(us), 1 [CLEANUP], MaxLatency(us), 1 [CLEANUP], 95thPercentileLatency(us), 1 [CLEANUP], 99thPercentileLatency(us), 1 [INSERT], Operations, 0 [INSERT], AverageLatency(us), NaN [INSERT], MinLatency(us), 9223372036854775807 [INSERT], MaxLatency(us), 0 [INSERT], 95thPercentileLatency(us), 0 [INSERT], 99thPercentileLatency(us), 0 [INSERT], Return=ERROR, 1 [INSERT-FAILED], Operations, 1 [INSERT-FAILED], AverageLatency(us), 72224.0 [INSERT-FAILED], MinLatency(us), 72192 [INSERT-FAILED], MaxLatency(us), 72255 [INSERT-FAILED], 95thPercentileLatency(us), 72255 [INSERT-FAILED], 99thPercentileLatency(us), 72255