Closed rodrigoalviani closed 8 years ago
Hi @rodrigoalviani ,
The log says that :
5935 [main] INFO org.elasticsearch.client.transport - [Mad Dog Rassitano] failed to get node info for {#transport#-1}{192.168.2.32}{192.168.2.32:9200}, disconnecting...
This issue is not related with mongolastic. Default transport port of es is 9300.
Hi @ozlerhakan,
Locally I connect to ES on 9200, where its is set?
Yes, but mongolastic communicates through es's tcp transport port.
do you solve your issue? @rodrigoalviani
@ozlerhakan Not solved, even with the door open I got the same error, telnet connect in 9300 smoothly. But anyway, I wrote an application that makes the transfer of MongoDB data for the ES. You can close, Thanks!
hello, I am sorry to tell that, but I have exactly the same problem :
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{xQcBppiDSRyq7Xm8YalPmg}{127.0.0.1}{127.0.0.1:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:360)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1247)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at com.kodcu.provider.ElasticToMongoProvider.getCount(ElasticToMongoProvider.java:39)
at com.kodcu.provider.Provider.transfer(Provider.java:16)
at com.kodcu.main.Mongolastic.proceedService(Mongolastic.java:69)
at java.util.Optional.ifPresent(Optional.java:159)
at com.kodcu.main.Mongolastic.start(Mongolastic.java:58)
at com.kodcu.main.Mongolastic.main(Mongolastic.java:35)
[INFO ] [2018-05-14 23:10:33] [cluster-ClusterId{value='5af9fb492dd4b341253b85cf', description='null'}-127.0.0.1:27017] [INFO]: - Exception in monitor thread while connecting to server 127.0.0.1:27017
com.mongodb.MongoSecurityException: Exception authenticating
at com.mongodb.connection.NativeAuthenticator.authenticate(NativeAuthenticator.java:48) ~[mongolastic.jar:?]
at com.mongodb.connection.InternalStreamConnectionInitializer.authenticateAll(InternalStreamConnectionInitializer.java:109) ~[mongolastic.jar:?]
at com.mongodb.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:46) ~[mongolastic.jar:?]
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:116) ~[mongolastic.jar:?]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) [mongolastic.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: com.mongodb.MongoCommandException: Command failed with error 18: 'auth failed' on server 127.0.0.1:27017. The full response is { "ok" : 0.0, "errmsg" : "auth failed", "code" : 18, "codeName" : "AuthenticationFailed" }
at com.mongodb.connection.CommandHelper.createCommandFailureException(CommandHelper.java:170) ~[mongolastic.jar:?]
at com.mongodb.connection.CommandHelper.receiveCommandResult(CommandHelper.java:123) ~[mongolastic.jar:?]
at com.mongodb.connection.CommandHelper.executeCommand(CommandHelper.java:32) ~[mongolastic.jar:?]
at com.mongodb.connection.NativeAuthenticator.authenticate(NativeAuthenticator.java:46) ~[mongolastic.jar:?]
... 5 more
My config file
misc:
dindex:
name: teleinfo
as: test_teleinfo
ctype:
name: conso
as: test_conso
direction: em
mongo:
host: 127.0.0.1
port: 27017
auth:
user: bber
pwd: "xxxxxxxx"
source: admin
mechanism: plain
elastic:
host: 127.0.0.1
port: 9300
dateFormat: "dd/MM/yyyy-HH:mm"
longToString: true
if it can helps I use ElasticSeach 5.5 Thanks for your help
Hi @MaxCreeger32 ,
Which auth do you use in your mongod? it says that { "ok" : 0.0, "errmsg" : "auth failed", "code" : 18, "codeName" : "AuthenticationFailed" }
sorry, yes I have seen that problem that is solved now : but I still have the ElasticSearch connection problem :
[INFO ] [2018-05-15 08:17:12] [main] [INFO]: -
Config Output:
{elastic=Elastic{host='127.0.0.1', port=9300, clusterName=null, dateFormat=dd/MM/yyyy-HH:mm, longToString=true, auth=null}, misc=Misc{batch=200, direction='em', dindex=Namespace{as='test_teleinfo', name='teleinfo'}, ctype=Namespace{as='test_conso', name='cons
o'}, dropDataset=true}, mongo=Mongo{host='127.0.0.1', port=27017, query='{}', project='null', auth=Auth [user=bber, pwd=xxxxxxx, source=admin, mechanism=plain]}}
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - no modules loaded
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - loaded plugin [org.elasticsearch.script.mustache.MustachePlugin]
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - loaded plugin [org.elasticsearch.transport.Netty4Plugin]
[INFO ] [2018-05-15 08:17:13] [main] [INFO]: - loaded plugin [org.elasticsearch.xpack.XPackPlugin]
[INFO ] [2018-05-15 08:17:14] [main] [INFO]: - Cluster created with settings {hosts=[127.0.0.1:27017], mode=MULTIPLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='5000 ms', maxWaitQueueSize=500}
[INFO ] [2018-05-15 08:17:14] [main] [INFO]: - Adding discovered server 127.0.0.1:27017 to client view of cluster
[INFO ] [2018-05-15 08:17:14] [main] [INFO]: - Load duration: 1394ms
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{JRfQErvgQ8edu5xPzAV4Qg}{127.0.0.1}{127.0.0.1:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:360)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1247)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at com.kodcu.provider.ElasticToMongoProvider.getCount(ElasticToMongoProvider.java:39)
at com.kodcu.provider.Provider.transfer(Provider.java:16)
at com.kodcu.main.Mongolastic.proceedService(Mongolastic.java:69)
at java.util.Optional.ifPresent(Optional.java:159)
at com.kodcu.main.Mongolastic.start(Mongolastic.java:58)
at com.kodcu.main.Mongolastic.main(Mongolastic.java:35)
Have you looked at these threads: https://stackoverflow.com/questions/25912572/java-elasticsearch-none-of-the-configured-nodes-are-available , https://stackoverflow.com/questions/42826735/nonodeavailableexception-none-of-the-configured-nodes-are-available?rq=1
you may need to specify a cluster name in the mongolastic config file:
...
elastic:
clusterName: elastictest
...
thanks for the links ! I have set the cluster name in the mongolastic file :
misc:
dindex:
name: teleinfo
as: test_teleinfo
ctype:
name: conso
as: test_conso
direction: em
mongo:
host: 127.0.0.1
port: 27017
auth:
user: bber
pwd: "cab32b79"
source: admin
mechanism: plain
elastic:
host: 127.0.0.1
port: 9300
dateFormat: "dd/MM/yyyy-HH:mm"
longToString: true
clusterName: elasticsearch
and the cluster name in my elastic search is correct :
$ curl http://127.0.0.1:9200
{
"name" : "vZtXodp",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "NEa7hb8cSPGbnYWU8hp87A",
"version" : {
"number" : "5.5.2",
"build_hash" : "b2f0c09",
"build_date" : "2017-08-14T12:33:14.154Z",
"build_snapshot" : false,
"lucene_version" : "6.6.0"
},
"tagline" : "You Know, for Search"
}
but I still have :
$ java -jar mongolastic.jar -f config.yml
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: -
Config Output:
{elastic=Elastic{host='127.0.0.1', port=9300, clusterName=elasticsearch, dateFormat=dd/MM/yyyy-HH:mm, longToString=true, auth=null}, misc=Misc{batch=200, direction='em', dindex=Namespace{as='test_teleinfo', name='teleinfo'}, ctype=Namespace{as='test_conso', n
ame='conso'}, dropDataset=true}, mongo=Mongo{host='127.0.0.1', port=27017, query='{}', project='null', auth=Auth [user=bber, pwd=xxxxxxxx, source=admin, mechanism=plain]}}
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - no modules loaded
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - loaded plugin [org.elasticsearch.script.mustache.MustachePlugin]
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - loaded plugin [org.elasticsearch.transport.Netty4Plugin]
[INFO ] [2018-05-15 12:43:13] [main] [INFO]: - loaded plugin [org.elasticsearch.xpack.XPackPlugin]
[INFO ] [2018-05-15 12:43:14] [main] [INFO]: - Cluster created with settings {hosts=[127.0.0.1:27017], mode=MULTIPLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='5000 ms', maxWaitQueueSize=500}
[INFO ] [2018-05-15 12:43:14] [main] [INFO]: - Adding discovered server 127.0.0.1:27017 to client view of cluster
[INFO ] [2018-05-15 12:43:14] [main] [INFO]: - Load duration: 1468ms
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{e2-U48vzS1y96o8z2A2llw}{127.0.0.1}{127.0.0.1:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:360)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1247)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at com.kodcu.provider.ElasticToMongoProvider.getCount(ElasticToMongoProvider.java:39)
at com.kodcu.provider.Provider.transfer(Provider.java:16)
at com.kodcu.main.Mongolastic.proceedService(Mongolastic.java:69)
at java.util.Optional.ifPresent(Optional.java:159)
at com.kodcu.main.Mongolastic.start(Mongolastic.java:58)
at com.kodcu.main.Mongolastic.main(Mongolastic.java:35)
as i see in your code, it seems to be somewhere in the elastic packages not yours ... i do not understand why I still have this error.
hmm, which version of mongolastic do you use? could you try previous jar files? maybe this is a problem for the latest jar
It seems to be the version 1.4.3 as the indication in the link is : https://github.com/ozlerhakan/mongolastic/releases/download/v1.4.3/mongolastic.jar
oh i just see in the release that there is a new one ! I will test with this one
It seems that the version 1.4.2 is OK ! 👍 I still have problems, but it is mongDb problem that has nothing connections with this problem (I did not create the target database 😝 ).
thanks for all your support
I have just seen that in tyhe histry of ElasticConfiguration.java :
- InetSocketTransportAddress ista = new InetSocketTransportAddress(InetAddress.getByName(config.getElastic().getHost()), config.getElastic().getPort());
+ TransportAddress ista = new TransportAddress(InetAddress.getByName(config.getElastic().getHost()), config.getElastic().getPort());
may be it is the TransportAdress class that have a problem in comparison of the InetSocketTransportAddress
Great @MaxCreeger32 ! good to hear that it works with 1.4.2 :)
may be it is the TransportAdress class that have a problem in comparison of the InetSocketTransportAddress
Hmm, if we have another case for that issue, I'll look a solution
Hi! I am observing the same problem for version 1.4.4 (btw, download path on main page pointing on 1.4.3)
My elastic instance info:
{
"name" : "0XzcHKY",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "1joCbUVHQPSPXCDlWj6izQ",
"version" : {
"number" : "6.2.4",
"build_hash" : "ccec39f",
"build_date" : "2018-04-12T20:37:28.497551Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
From elastic starting log I can see line:
[2018-09-11T18:12:48,816][INFO ][o.e.t.TransportService ] [0XzcHKY] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
I have also attached my docker-compose.yml
and mongolastic
config
My error looks like:
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: -
Config Output:
{elastic=Elastic{host='127.0.0.1', port=9300, clusterName=docker-cluster, dateFormat=null, longToString=false, auth=null}, misc=Misc{batch=200, direction='me', dindex=Namespace{as='null', name='api'}, ctype=Namespace{as='null', name='conversations'}, dropDataset=false}, mongo=Mongo{host='0.0.0.0', port=27017, query='{}', project='null', auth=null}}
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: - no modules loaded
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: - loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: - loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: - loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: - loaded plugin [org.elasticsearch.script.mustache.MustachePlugin]
[INFO ] [2018-09-11 14:12:53] [main] [INFO]: - loaded plugin [org.elasticsearch.transport.Netty4Plugin]
[INFO ] [2018-09-11 14:12:56] [main] [INFO]: - Cluster created with settings {hosts=[0.0.0.0:27017], mode=MULTIPLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='5000 ms', maxWaitQueueSize=500}
[INFO ] [2018-09-11 14:12:56] [main] [INFO]: - Adding discovered server 0.0.0.0:27017 to client view of cluster
[INFO ] [2018-09-11 14:12:56] [main] [INFO]: - No server chosen by ReadPreferenceServerSelector{readPreference=ReadPreference{name=primaryPreferred}} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=0.0.0.0:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 5000 ms before timing out
[INFO ] [2018-09-11 14:12:56] [cluster-ClusterId{value='5b9805a85e14334ebbdad732', description='null'}-0.0.0.0:27017] [INFO]: - Opened connection [connectionId{localValue:1, serverValue:85}] to 0.0.0.0:27017
[INFO ] [2018-09-11 14:12:56] [cluster-ClusterId{value='5b9805a85e14334ebbdad732', description='null'}-0.0.0.0:27017] [INFO]: - Monitor thread successfully connected to server with description ServerDescription{address=0.0.0.0:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 2]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, roundTripTimeNanos=7181257}
[INFO ] [2018-09-11 14:12:56] [cluster-ClusterId{value='5b9805a85e14334ebbdad732', description='null'}-0.0.0.0:27017] [INFO]: - Discovered cluster type of STANDALONE
[INFO ] [2018-09-11 14:12:56] [main] [INFO]: - Opened connection [connectionId{localValue:2, serverValue:86}] to 0.0.0.0:27017
[INFO ] [2018-09-11 14:12:56] [main] [INFO]: - Mongo collection count: 1
[INFO ] [2018-09-11 14:12:56] [main] [INFO]: - Transferring data began to elasticsearch.
[WARN ] [2018-09-11 14:12:57] [main] [WARN]: - Failed to execute bulk request 1.
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{UiDmeqXiREmHZ9AiHB8SAg}{127.0.0.1}{127.0.0.1:9300}]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:371) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java:482) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.Retry$RetryHandler.execute(Retry.java:206) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.Retry.withBackoff(Retry.java:64) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.BulkRequestHandler.execute(BulkRequestHandler.java:65) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.BulkProcessor.execute(BulkProcessor.java:339) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.BulkProcessor.awaitClose(BulkProcessor.java:239) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.service.ElasticBulkService.close(ElasticBulkService.java:79) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.main.Mongolastic.lambda$proceedService$0(Mongolastic.java:70) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.provider.Provider.transfer(Provider.java:35) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.main.Mongolastic.proceedService(Mongolastic.java:69) ~[mongolastic.1.4.4.jar:?]
at java.util.Optional.ifPresent(Optional.java:172) [?:?]
at com.kodcu.main.Mongolastic.start(Mongolastic.java:58) [mongolastic.1.4.4.jar:?]
at com.kodcu.main.Mongolastic.main(Mongolastic.java:35) [mongolastic.1.4.4.jar:?]
[ERROR] [2018-09-11 14:12:57] [main] [ERROR]: - Transfer failed.
[ERROR] [2018-09-11 14:12:57] [main] [ERROR]: - None of the configured nodes are available: [{#transport#-1}{UiDmeqXiREmHZ9AiHB8SAg}{127.0.0.1}{127.0.0.1:9300}]
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{UiDmeqXiREmHZ9AiHB8SAg}{127.0.0.1}{127.0.0.1:9300}]
at com.kodcu.listener.BulkProcessorListener.afterBulk(BulkProcessorListener.java:33) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.BulkRequestHandler.execute(BulkRequestHandler.java:96) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.BulkProcessor.execute(BulkProcessor.java:339) ~[mongolastic.1.4.4.jar:?]
at org.elasticsearch.action.bulk.BulkProcessor.awaitClose(BulkProcessor.java:239) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.service.ElasticBulkService.close(ElasticBulkService.java:79) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.main.Mongolastic.lambda$proceedService$0(Mongolastic.java:70) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.provider.Provider.transfer(Provider.java:35) ~[mongolastic.1.4.4.jar:?]
at com.kodcu.main.Mongolastic.proceedService(Mongolastic.java:69) ~[mongolastic.1.4.4.jar:?]
at java.util.Optional.ifPresent(Optional.java:172) [?:?]
at com.kodcu.main.Mongolastic.start(Mongolastic.java:58) [mongolastic.1.4.4.jar:?]
at com.kodcu.main.Mongolastic.main(Mongolastic.java:35) [mongolastic.1.4.4.jar:?]
[INFO ] [2018-09-11 14:12:57] [main] [INFO]: - Closed connection [connectionId{localValue:2, serverValue:86}] to 0.0.0.0:27017 because the pool has been closed.
[INFO ] [2018-09-11 14:12:57] [main] [INFO]: - Load duration: 3746ms
My config file:
Error: