search

brianfrankcooper / YCSB

Yahoo! Cloud Serving Benchmark
Apache License 2.0
4.96k stars 2.25k forks source link

YCSB mongodb-async to report the following #996

Open liu-shaobo opened 7 years ago

liu-shaobo commented 7 years ago

hi，all Use YCSB mongodb-async to report the following，please help me。

use mongo normal： ./mongo --host 10.9.137.251 ycsb -u user -p passwd MongoDB shell version: 3.0.15 connecting to: 10.9.137.251:27017/ycsb

run command: ./bin/ycsb load mongodb-async -P workloads/workloada -s -p mongodb.url=mongodb://user:passwd@10.9.137.251:27017/ycsb?w=0

workloads file config: recordcount=10000 operationcount=10000 workload=com.yahoo.ycsb.workloads.CoreWorkload readallfields=true readproportion=0.5 updateproportion=0.5 scanproportion=0 insertproportion=0 requestdistribution=zipfian threadcount=10

error log: 11:46:37.544 [Thread-12] WARN c.a.m.c.c.a.AuthenticatingConnection - Authentication failed: [] com.allanbank.mongodb.error.MongoDbAuthenticationException: com.allanbank.mongodb.error.ReplyException: auth failed at com.allanbank.mongodb.client.connection.auth.MongoDbAuthenticator.result(MongoDbAuthenticator.java:123) ~[mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.auth.AuthenticatingConnection.ensureAuthenticated(AuthenticatingConnection.java:181) ~[mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.auth.AuthenticatingConnection.send(AuthenticatingConnection.java:133) ~[mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.bootstrap(BootstrapConnectionFactory.java:179) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.createDelegate(BootstrapConnectionFactory.java:290) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.getDelegate(BootstrapConnectionFactory.java:268) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.getClusterStats(BootstrapConnectionFactory.java:114) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.ClientImpl.getClusterStats(ClientImpl.java:199) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.AbstractMongoOperations.doInsertAsync(AbstractMongoOperations.java:1037) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.AbstractMongoOperations.insertAsync(AbstractMongoOperations.java:583) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.AbstractAsyncMongoCollection.insertAsync(AbstractAsyncMongoCollection.java:1255) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.SynchronousMongoCollectionImpl.insert(SynchronousMongoCollectionImpl.java:684) [mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.SynchronousMongoCollectionImpl.insert(SynchronousMongoCollectionImpl.java:722) [mongodb-async-driver-2.0.1.jar:na] at com.yahoo.ycsb.db.AsyncMongoDbClient.insert(AsyncMongoDbClient.java:272) [mongodb-binding-0.13.0-SNAPSHOT.jar:na] at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:220) [core-0.13.0-SNAPSHOT.jar:na] at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:562) [core-0.13.0-SNAPSHOT.jar:na] at com.yahoo.ycsb.ClientThread.run(Client.java:459) [core-0.13.0-SNAPSHOT.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80] Caused by: java.util.concurrent.ExecutionException: com.allanbank.mongodb.error.ReplyException: auth failed at com.allanbank.mongodb.client.FutureCallback$Sync.getValue(FutureCallback.java:581) ~[mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.FutureCallback$Sync.get(FutureCallback.java:489) ~[mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.FutureCallback.get(FutureCallback.java:223) ~[mongodb-async-driver-2.0.1.jar:na] at com.allanbank.mongodb.client.connection.auth.MongoDbAuthenticator.result(MongoDbAuthenticator.java:117) ~[mongodb-async-driver-2.0.1.jar:na] ... 17 common frames omitted

allanbank commented 7 years ago

What version of MongoDB are you using and have you verified via the mongo shell that the user name and password are correct?

liu-shaobo commented 7 years ago

yes ，mongo shell login correct。

allanbank commented 7 years ago

What version of MongoDB are you using?

T0FFF commented 7 years ago

Hello, I have the same problem. I use MongoDB 3.4.9. The following command works: ./bin/ycsb load mongodb -P workloads/workloada -s -p mongodb.url=mongodb://user:passwd@x.x.x.x:27017/ycsb?w=0

hongkailiu commented 6 years ago

3.2.10 Hit the same issue. mongodb works but mongodb-async got

DBWrapper: report latency for each error is false and specific error codes to track for latency are: []
17:59:32.920 [Thread-3] WARN  c.a.m.c.c.a.AuthenticatingConnection - Authentication failed: []
com.allanbank.mongodb.error.MongoDbAuthenticationException: com.allanbank.mongodb.error.ReplyException: auth failed
    at com.allanbank.mongodb.client.connection.auth.MongoDbAuthenticator.result(MongoDbAuthenticator.java:123) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.auth.AuthenticatingConnection.ensureAuthenticated(AuthenticatingConnection.java:181) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.auth.AuthenticatingConnection.send(AuthenticatingConnection.java:133) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.bootstrap(BootstrapConnectionFactory.java:179) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.createDelegate(BootstrapConnectionFactory.java:290) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.getDelegate(BootstrapConnectionFactory.java:268) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.getClusterStats(BootstrapConnectionFactory.java:114) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.ClientImpl.getClusterStats(ClientImpl.java:199) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.AbstractMongoOperations.doInsertAsync(AbstractMongoOperations.java:1037) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.AbstractMongoOperations.insertAsync(AbstractMongoOperations.java:583) [mongodb-async-driver-2.0.1.jar:na]
allanbank commented 6 years ago

How do you have authentication setup on the server side? MongoDB CR or SCRAM?

hongkailiu commented 6 years ago 
MongoDB shell version: 3.2.10
connecting to: 127.0.0.1:27017/admin
...
> db.system.users.findOne({user: "***"})
{
    "_id" : "testdb.***",
    "user" : "***",
    "db" : "testdb",
    "credentials" : {
        "SCRAM-SHA-1" : {
            "iterationCount" : 10000,
            "salt" : "***",
            "storedKey" : "***",
            "serverKey" : "***"
        }
    },
    "roles" : [
        {
            "role" : "readWrite",
            "db" : "testdb"
        }
    ]
}
allanbank commented 6 years ago

Thanks, that is what I needed to try and reproduce the issue. I probably won't get to it until this weekend.

Rob.

hongkailiu commented 6 years ago

Thanks a lot, Rob.

Hongkai

busbey commented 6 years ago

folks still hitting this?

hongkailiu commented 6 years ago

@busbey It is still there when using 0.14.0 version of ycsb. Same mongodb version as before: MongoDB shell version: 3.2.10

$ ./bin/ycsb load mongodb-async -s -P workloads/workloada -p mongodb.url=mongodb://u:p@<ip>:27017/testdb
/docker-java-home/bin/java -cp /ycsb/mongodb-binding/conf:/ycsb/conf:/ycsb/lib/HdrHistogram-2.1.4.jar:/ycsb/lib/core-0.14.0.jar:/ycsb/lib/htrace-core4-4.1.0-incubating.jar:/ycsb/lib/jackson-core-asl-1.9.4.jar:/ycsb/lib/jackson-mapper-asl-1.9.4.jar:/ycsb/mongodb-binding/lib/logback-classic-1.1.2.jar:/ycsb/mongodb-binding/lib/logback-core-1.1.2.jar:/ycsb/mongodb-binding/lib/mongo-java-driver-3.6.3.jar:/ycsb/mongodb-binding/lib/mongodb-async-driver-2.0.1.jar:/ycsb/mongodb-binding/lib/mongodb-binding-0.14.0.jar:/ycsb/mongodb-binding/lib/slf4j-api-1.7.25.jar:/ycsb/mongodb-binding/lib/snappy-java-1.1.7.1.jar com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.AsyncMongoDbClient -s -P workloads/workloada -p mongodb.url=mongodb://redhat:redhat@172.24.89.134:27017/testdb -load
Command line: -db com.yahoo.ycsb.db.AsyncMongoDbClient -s -P workloads/workloada -p mongodb.url=mongodb://redhat:redhat@172.24.89.134:27017/testdb -load
YCSB Client 0.14.0

Loading workload...
Starting test.
2018-07-10 20:07:00:143 0 sec: 0 operations; est completion in 0 second 
mongo connection created with mongodb://redhat:redhat@172.24.89.134:27017/testdb
DBWrapper: report latency for each error is false and specific error codes to track for latency are: []
20:07:00.257 [Thread-3] WARN  c.a.m.c.c.a.AuthenticatingConnection - Authentication failed: []
com.allanbank.mongodb.error.MongoDbAuthenticationException: com.allanbank.mongodb.error.ReplyException: auth failed
    at com.allanbank.mongodb.client.connection.auth.MongoDbAuthenticator.result(MongoDbAuthenticator.java:123) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.auth.AuthenticatingConnection.ensureAuthenticated(AuthenticatingConnection.java:181) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.auth.AuthenticatingConnection.send(AuthenticatingConnection.java:133) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.bootstrap(BootstrapConnectionFactory.java:179) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.createDelegate(BootstrapConnectionFactory.java:290) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.getDelegate(BootstrapConnectionFactory.java:268) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.bootstrap.BootstrapConnectionFactory.getClusterStats(BootstrapConnectionFactory.java:114) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.ClientImpl.getClusterStats(ClientImpl.java:199) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.AbstractMongoOperations.doInsertAsync(AbstractMongoOperations.java:1037) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.AbstractMongoOperations.insertAsync(AbstractMongoOperations.java:583) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.AbstractAsyncMongoCollection.insertAsync(AbstractAsyncMongoCollection.java:1255) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.SynchronousMongoCollectionImpl.insert(SynchronousMongoCollectionImpl.java:684) [mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.SynchronousMongoCollectionImpl.insert(SynchronousMongoCollectionImpl.java:722) [mongodb-async-driver-2.0.1.jar:na]
    at com.yahoo.ycsb.db.AsyncMongoDbClient.insert(AsyncMongoDbClient.java:272) [mongodb-binding-0.14.0.jar:na]
    at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:221) [core-0.14.0.jar:na]
    at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:562) [core-0.14.0.jar:na]
    at com.yahoo.ycsb.ClientThread.run(Client.java:468) [core-0.14.0.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
Caused by: java.util.concurrent.ExecutionException: com.allanbank.mongodb.error.ReplyException: auth failed
    at com.allanbank.mongodb.client.FutureCallback$Sync.getValue(FutureCallback.java:581) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.FutureCallback$Sync.get(FutureCallback.java:489) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.FutureCallback.get(FutureCallback.java:223) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.auth.MongoDbAuthenticator.result(MongoDbAuthenticator.java:117) ~[mongodb-async-driver-2.0.1.jar:na]
    ... 17 common frames omitted
Caused by: com.allanbank.mongodb.error.ReplyException: auth failed
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.asError(AbstractValidatingReplyCallback.java:197) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.asError(AbstractValidatingReplyCallback.java:216) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.asError(AbstractValidatingReplyCallback.java:141) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.asError(AbstractValidatingReplyCallback.java:112) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.checkForError(AbstractValidatingReplyCallback.java:244) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.verify(AbstractValidatingReplyCallback.java:298) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.callback(AbstractValidatingReplyCallback.java:90) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.AbstractValidatingReplyCallback.callback(AbstractValidatingReplyCallback.java:51) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.ReplyHandler.run(ReplyHandler.java:127) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.callback.ReplyHandler.reply(ReplyHandler.java:96) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.socket.AbstractSocketConnection.reply(AbstractSocketConnection.java:632) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.socket.AbstractSocketConnection.doReceiveOne(AbstractSocketConnection.java:509) ~[mongodb-async-driver-2.0.1.jar:na]
    at com.allanbank.mongodb.client.connection.socket.ReceiveRunnable.run(ReceiveRunnable.java:59) ~[mongodb-async-driver-2.0.1.jar:na]
    ... 1 common frames omitted
20:07:00.275 [Thread-3] WARN  c.a.m.c.c.a.AuthenticatingConnection - Authentication failed: []
com.allanbank.mongodb.error.MongoDbAuthenticationException: com.allanbank.mongodb.error.ReplyException: auth failed
busbey commented 6 years ago

thanks for confirming @hongkailiu

rivetgeekwil commented 6 years ago

I'm still getting the same error, mongodb 4 against a replica set, this is the connection string:

./bin/ycsb load mongodb-async -s -P workloads/workloada -p mongodb.url=mongodb://username:pw@node1.gcp.mongodb.net:27017,node2.gcp.mongodb.net:27017,node3.gcp.mongodb.net:27017/ycsb?ssl=true&replicaSet=DevCluster&authSource=admin&retryWrites=true&readPreference=primary

Exact same error as posted above. We are using SCRAM for auth.