elastic / stack-docker

Project no longer maintained.
Apache License 2.0
1.18k stars 450 forks source link

Cant load dataset #61

Closed Rajashreegr closed 5 years ago

Rajashreegr commented 5 years ago

I set up the elastic search and all the containers are up & running.

But I'm facing an issue while loading the dataset to elastic search. I setup the mappings & trying to load dataset [ I'm following the Tutorial - https://www.elastic.co/guide/en/kibana/current/tutorial-load-dataset.html.]

On running the curl upload command - curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.json, I'm getting the error 'curl: (52) Empty reply from server'.

On listing the docker containers it shows the below containers are running successfully. docker.elastic.co/kibana/kibana:6.3.0 "/usr/local/bin/kiba…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:5601->5601/tcp kibana a3aa8556c225 docker.elastic.co/logstash/logstash:6.3.0 "/usr/local/bin/dock…" 2 hours ago Up 2 hours (healthy) 5044/tcp, 9600/tcp logstash 79c75ecde762 docker.elastic.co/elasticsearch/elasticsearch:6.3.0 "/usr/local/bin/dock…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:9200->9200/tcp, 9300/tcp elasticsearch

But when I checked for [http://localhost:9200/bank ] it is not up & running. Not sure what is wrong here.

fxdgear commented 5 years ago

@Rajashreegr soo it looks to me like stack-docker is functioning properly and as intended.

The bigger issue is getting your dataset into Elasticesarch.

So can you check a few things for me...

1: navigate to localhost:9200 in your browser and login with the username/password elastic/`

  1. navigate to localhost:5600 and use the same username and password.

This is going to check that es and kibana are running and accessible via your host machine.

If both of those work, I'd suggest trying a few commands in the kibana console to try creating the index you wanna bulk index into delete the index.

if that works we well then we need to try doing a "bulk" load via kibana console (small subset of your data maybe 2-3 lines of your accounts.json file.

If that works then I'm going to say we need to look at the accounts.json and see if it's maybe too big for the size of the cluster..

ErikWallin commented 5 years ago

Have you managed to turn off ssl? I have to use https.

fxdgear commented 5 years ago

@erikwallin is right. You’re not using ssl in your curl. I didn’t see that initially.

Rajashreegr commented 5 years ago

Let me check those two things and see what I have missed. Thanks, @fxdgear & @ErikWallin

fxdgear commented 5 years ago

When you do the tests with the browser and elasticsearch make sure you use https

Kibana is http.

On Mon, Oct 22, 2018 at 16:24 Rajashree notifications@github.com wrote:

Let me check those two things and see what I have missed. Thanks, @fxdgear https://github.com/fxdgear & @ErikWallin https://github.com/ErikWallin

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/elastic/stack-docker/issues/61#issuecomment-432012280, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI8dqYKluEhSyFoQ6waXMhvGqaNykqVks5unkWBgaJpZM4XyRwd .

Rajashreegr commented 5 years ago

I disabled the xpack ssl security in elasticsearch.yml file. xpack.security.enabled: true xpack.security.http.ssl.enabled: false xpack.security.transport.ssl.enabled: false

Now the elastic search started properly but facing some other issue. When I run the curl command on the Elastic search URL - http://localhost:9200, I'm getting the following output. { "name" : "xitm0Cn", "cluster_name" : "docker-cluster", "cluster_uuid" : "9N0lMgLVSWmipPgThJ_JoA", "version" : { "number" : "6.3.0", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "424e937", "build_date" : "2018-06-11T23:38:03.357887Z", "build_snapshot" : false, "lucene_version" : "7.3.1", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }

Now the Kibana & logstash are not able to reach elastic search and keep waiting for its connection.

Rajashreegr commented 5 years ago

If I do not disable the ssl, the elastic search is not getting started and keep failing with the issue 'java.io.StreamCorruptedException: invalid internal transport message format'.

fxdgear commented 5 years ago

@Rajashreegr

So lets go back and start at the beginning.

all you have to do to get the data set to load is run

curl -H 'Content-Type: application/x-ndjson' -u 'elastic:<your_elastic_password>' -k -XPOST 'https://localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.jso

You are making this too hard by trying to edit all the settings. Just leave SSL turned on. You don't want to run HTTP services without it anyway.

fxdgear commented 5 years ago

Closing. @Rajashreegr if you're continue to having issues after using https in your curl to load the data please re-open the issue. Thanks