search

brianfrankcooper / YCSB

Yahoo! Cloud Serving Benchmark
Apache License 2.0
4.96k stars 2.25k forks source link

Exception using YCSB with Elasticsearch (ver. 5.x.x) binding in remote mode #1012

Closed Shortyxxy closed 7 years ago

Shortyxxy commented 7 years ago

Hello, i have a problem using the Elasticsearch binding in remote mode. I tried the following command with the Elasticsearch versions 5.5.1, 5.5.2 and 6.0.0 beta and the YCSB version 0.12.0 as well as a snapshot from the current master and the risdenk fork (28.08.17):

./bin/ycsb.sh load elasticsearch -P workloads/workloada -p es.remote=true -p path.home=/usr/share/elasticsearch -s

I configured the elasticsearch.yml with the following line:

cluster.name: es.ycsb.cluster

It works perfectly with Elasticsearch version 2.2.3 according to the examples in the elasticsearch binding folder as well as the answer in the following issue from jasontedor.

On the described versions i get the following error message:

YCSB Client 0.12.0
Command line: -db com.yahoo.ycsb.db.ElasticsearchClient -P workloads/workloada -p es.remote=true -p path.home=/usr/share/elasticsearch -s -load
Loading workload...
Starting test.
2017-08-28 14:43:20:938 0 sec: 0 operations; est completion in 0 seconds 
Elasticsearch starting node = es.ycsb.cluster
Elasticsearch node path.home = /usr/share/elasticsearch
Elasticsearch Remote Mode = true
Elasticsearch Remote Hosts = localhost:9300
Aug 28, 2017 2:43:21 PM org.elasticsearch.plugins.PluginsService <init>
INFORMATION: [Chen Lu] modules [], plugins [], sites []
Aug 28, 2017 2:43:21 PM org.elasticsearch.client.transport.TransportClientNodesService$SniffNodesSampler$1$1 handleException
INFORMATION: [Chen Lu] failed to get local cluster state for {#transport#-1}{127.0.0.1}{127.0.0.1:9300}, disconnecting...
NodeDisconnectedException[[][127.0.0.1:9300][cluster:monitor/state] disconnected]

Exception in thread "Thread-3" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{127.0.0.1}{127.0.0.1:9300}]]
    at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)
    at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
    at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1221)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.exists(AbstractClient.java:1241)
    at com.yahoo.ycsb.db.ElasticsearchClient.init(ElasticsearchClient.java:146)
    at com.yahoo.ycsb.DBWrapper.init(DBWrapper.java:99)
    at com.yahoo.ycsb.ClientThread.run(Client.java:418)
    at java.lang.Thread.run(Thread.java:745)
2017-08-28 14:43:21:806 0 sec: 0 operations; est completion in 106751991167300 days 15 hours 
[OVERALL], RunTime(ms), 898.0
[OVERALL], Throughput(ops/sec), 0.0
[TOTAL_GCS_PS_Scavenge], Count, 1.0
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 68.0
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 7.57238307349666
[TOTAL_GCS_PS_MarkSweep], Count, 0.0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0.0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 1.0
[TOTAL_GC_TIME], Time(ms), 68.0
[TOTAL_GC_TIME_%], Time(%), 7.57238307349666

I also wanted to try the elasticsearch5 binding like described in this README commited from the risdenk fork, but it seems not to be recognized from the ycsb script, since it is not implemented as an option yet.

ycsb: error: argument database: invalid choice: 'elasticsearch5' (choose from 'accumulo', 'aerospike', 'arangodb', 'asynchbase', 'basic', 'cassandra-cql', 'cassandra2-cql', 'couchbase', 'couchbase2', 'dynamodb', 'elasticsearch', 'geode', 'googlebigtable', 'googledatastore', 'hbase094', 'hbase098', 'hbase10', 'hypertable', 'infinispan', 'infinispan-cs', 'jdbc', 'kudu', 'mapkeeper', 'memcached', 'mongodb', 'mongodb-async', 'nosqldb', 'orientdb', 'rados', 'redis', 'rest', 'riak', 's3', 'solr', 'solr6', 'tarantool', 'voldemort')

I really would be glad for some help regarding this issue. I already looked up the answers to the issues #856 , #796 but setting client.transport.sniff=false didn't work also (i'm not using docker). Embedded mode works fine in all versions that i've tested.

If you need more details just ask me!

levaphenyl commented 7 years ago

To use the elasticsearch5 flag, add the following line in bin/yscb, in dictionary DATABASES: "elasticsearch5": "com.yahoo.ycsb.db.elasticsearch5.ElasticsearchClient",

Shortyxxy commented 7 years ago

Thank you very much @levaphenyl for the instruction! It is actually working now with the elasticsearch5 flag and Elasticsearch version 5.5.1 as well as the new 5.5.2 (it is not working with 6.0.0 but its just a beta so we shouldn't really care). It tried it on a local machine as well as with a remote call to another server with a running Elasticsearch instance, both works, so the issue can be closed from my side!