Kibana 4 can't connect to local Elasticsearch

mclarke47 commented 9 years ago

I am trying to set up Kibana 4 with Elasticsearch 1.4.4. However everytime I start kibana all that is displayed is the Kibana header and a blank screen.

The console prints the following error message:

"INFO: 2015-02-20T10:53:52Z Adding connection to http://localhost:5601/elasticsearch

" index.js:81071 bootstrap index.js:45691 es check index.js:45691 config init index.js:45691 "Error: Unknown error while connecting to Elasticsearch ErrorAbstract@http://localhost:5601/index.js?_b=5827:80209:5 Generic@http://localhost:5601/index.js?_b=5827:80241:3 respond@http://localhost:5601/index.js?_b=5827:81522:9 checkRespForFailure@http://localhost:5601/index.js?_b=5827:81488:15 [198]</AngularConnector.prototype.request/<@http://localhost:5601/index.js?_b=5827:80157:7 qFactory/defer/deferred.promise.then/wrappedErrback@http://localhost:5601/index.js?_b=5827:20882:15 qFactory/defer/deferred.promise.then/wrappedErrback@http://localhost:5601/index.js?_b=5827:20882:15 qFactory/defer/deferred.promise.then/wrappedErrback@http://localhost:5601/index.js?_b=5827:20882:15 qFactory/createInternalRejectedPromise/<.then/<@http://localhost:5601/index.js?_b=5827:21015:13 $RootScopeProvider/this.$get</Scope.prototype.$eval@http://localhost:5601/index.js?_b=5827:22002:9 $RootScopeProvider/this.$get</Scope.prototype.$digest@http://localhost:5601/index.js?_b=5827:21814:15 $RootScopeProvider/this.$get</Scope.prototype.$apply@http://localhost:5601/index.js?_b=5827:22106:13 done@http://localhost:5601/index.js?_b=5827:17641:34 completeRequest@http://localhost:5601/index.js?_b=5827:17855:7 createHttpBackend/</xhr.onreadystatechange@http://localhost:5601/index.js?_b=5827:17798:1 " index.js:19317

The kibana startup script prints the following:

{"@timestamp":"2015-02-20T11:02:59.885Z","level":"info","message":"Listening on 0.0.0.0:5601","node_env":"production"} {"@timestamp":"2015-02-20T11:03:04.844Z","level":"info","message":"GET / 304 - 10ms","node_env":"production","request":{"method":"GET","url":"/","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,/_;q=0.8","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"640-743943594\"","cache-control":"max-age=0"},"remoteAddress":"127.0.0.1","remotePort":52750},"response":{"statusCode":304,"responseTime":10,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:04.869Z","level":"info","message":"GET /styles/main.css?_b=5827 304 - 2ms","node_env":"production","request":{"method":"GET","url":"/styles/main.css?_b=5827","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"text/css,/_;q=0.1","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"31b9e-743943594\"","cache-control":"max-age=0"},"remoteAddress":"127.0.0.1","remotePort":52750},"response":{"statusCode":304,"responseTime":2,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:04.873Z","level":"info","message":"GET /bower_components/requirejs/require.js?_b=5827 304 - 3ms","node_env":"production","request":{"method":"GET","url":"/bower_components/requirejs/require.js?_b=5827","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"/_","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"1448b-743943594\"","cache-control":"max-age=0"},"remoteAddress":"127.0.0.1","remotePort":52750},"response":{"statusCode":304,"responseTime":3,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:04.875Z","level":"info","message":"GET /require.config.js?_b=5827 304 - 2ms","node_env":"production","request":{"method":"GET","url":"/require.config.js?_b=5827","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"/_","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"939-743943594\"","cache-control":"max-age=0"},"remoteAddress":"127.0.0.1","remotePort":52751},"response":{"statusCode":304,"responseTime":2,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:04.876Z","level":"info","message":"GET /images/initial_load.gif 304 - 1ms","node_env":"production","request":{"method":"GET","url":"/images/initial_load.gif","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"image/png,image/;q=0.8,/;q=0.5","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"2e9e-743943594\"","cache-control":"max-age=0"},"remoteAddress":"127.0.0.1","remotePort":52752},"response":{"statusCode":304,"responseTime":1,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:04.942Z","level":"info","message":"GET /index.js?_b=5827 304 - 2ms","node_env":"production","request":{"method":"GET","url":"/index.js?_b=5827","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"/_","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"4eb1c0-743943594\""},"remoteAddress":"127.0.0.1","remotePort":52753},"response":{"statusCode":304,"responseTime":2,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:05.225Z","level":"info","message":"GET /config?_b=5827 304 - 3ms","node_env":"production","request":{"method":"GET","url":"/config?_b=5827","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,/_;q=0.8","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-none-match":"W/\"152-1d5b248a\""},"remoteAddress":"127.0.0.1","remotePort":52750},"response":{"statusCode":304,"responseTime":3,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:05.469Z","level":"info","message":"GET /images/no_border.png 304 - 1ms","node_env":"production","request":{"method":"GET","url":"/images/no_border.png","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"image/png,image/;q=0.8,/;q=0.5","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/styles/main.css?_b=5827","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive","if-modified-since":"Thu, 12 Feb 2015 19:31:00 GMT","if-none-match":"W/\"14f-743943594\"","cache-control":"max-age=0"},"remoteAddress":"127.0.0.1","remotePort":52751},"response":{"statusCode":304,"responseTime":1,"contentLength":0}} {"@timestamp":"2015-02-20T11:03:06.501Z","level":"info","message":"GET / 502 - 1026ms","node_env":"production","request":{"method":"GET","url":"/elasticsearch/","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"application/json, text/plain, /_","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","cookie":"splunkweb_csrf_token_8000=7478925270992322182","connection":"keep-alive"},"remoteAddress":"127.0.0.1","remotePort":52752},"response":{"statusCode":502,"responseTime":1026,"contentLength":0}}

I had seen connection issues due to cors and so edited my /etc/elasticsearch/elasticsearch.yml to the following: http.cors.enabled: true http.cors.allow-origin: "/.*/"

I am using cntlm as a proxy and when I navigate to http://localhost:5601/elasticsearch I see the following cntlm error: "502 Connection refused Cntlm proxy failed to complete the request."

Is there extra set up for kibana 4/elasticsearch-1.4.4 as opposed to kibana 3 when using behind a proxy? Kibana 3 works just fine for me with my current config.

xavierromero commented 9 years ago

I have exactly the same problem, but in my case elasticsearch is not in the local machine. Also using elasticserch 1.4.4

From the same machine I'm running kibana3 with no problem at all.

xavierromero commented 9 years ago

My kibana4 console output:

[15:05:52][root@nex-es_dashboard-01 kibana-4.0.0-linux-x64]# ./bin/kibana {"@timestamp":"2015-02-20T14:06:06.206Z","level":"info","message":"Listening on 0.0.0.0:5601","node_env":"production"} {"@timestamp":"2015-02-20T14:06:24.264Z","level":"info","message":"GET / 304 - 8ms","nodeenv":"production","request":{"method":"GET","url":"/","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/_;q=0.8","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)","if-none-match":"W/\"63c-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT"},"remoteAddress":"10.50.0.3","remotePort":55927},"response":{"statusCode":304,"responseTime":8,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:24.285Z","level":"info","message":"GET /styles/main.css?_b=5888 304 - 3ms","node_env":"production","request":{"method":"GET","url":"/styles/main.css?b=5888","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"text/css,/_;q=0.1","if-none-match":"W/\"324ce-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)"},"remoteAddress":"10.50.0.3","remotePort":55927},"response":{"statusCode":304,"responseTime":3,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:24.285Z","level":"info","message":"GET /bower_components/requirejs/require.js?_b=5888 304 - 3ms","node_env":"production","request":{"method":"GET","url":"/bower_components/requirejs/require.js?b=5888","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"/_","if-none-match":"W/\"14640-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)"},"remoteAddress":"10.50.0.3","remotePort":55929},"response":{"statusCode":304,"responseTime":3,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:24.286Z","level":"info","message":"GET /images/initial_load.gif 304 - 3ms","node_env":"production","request":{"method":"GET","url":"/images/initialload.gif","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"image/webp,/_;q=0.8","if-none-match":"W/\"2e9e-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)"},"remoteAddress":"10.50.0.3","remotePort":55926},"response":{"statusCode":304,"responseTime":3,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:24.287Z","level":"info","message":"GET /require.config.js?_b=5888 304 - 4ms","node_env":"production","request":{"method":"GET","url":"/require.config.js?b=5888","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"/_","if-none-match":"W/\"939-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)"},"remoteAddress":"10.50.0.3","remotePort":55928},"response":{"statusCode":304,"responseTime":4,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:24.341Z","level":"info","message":"GET /index.js?_b=5888 304 - 2ms","node_env":"production","request":{"method":"GET","url":"/index.js?b=5888","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"/_","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)","if-none-match":"W/\"4eee73-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT"},"remoteAddress":"10.50.0.3","remotePort":55928},"response":{"statusCode":304,"responseTime":2,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:24.921Z","level":"info","message":"GET /config?_b=5888 304 - 2ms","node_env":"production","request":{"method":"GET","url":"/config?b=5888","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","accept":"/_","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)","if-none-match":"W/\"152-40215675\""},"remoteAddress":"10.50.0.3","remotePort":55928},"response":{"statusCode":304,"responseTime":2,"contentLength":0}} {"@timestamp":"2015-02-20T14:06:25.151Z","level":"info","message":"GET /images/no_border.png 304 - 1ms","node_env":"production","request":{"method":"GET","url":"/images/noborder.png","headers":{"host":"kibana3.datacenter.nexica.com:5601","connection":"keep-alive","accept":"image/webp,/_;q=0.8","if-none-match":"W/\"10ab-1491436542\"","if-modified-since":"Thu, 19 Feb 2015 16:10:01 GMT","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36","referer":"http://kibana3.datacenter.nexica.com:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","cookie":"COMPANY_ID=10157; ID=3061494a724379574568553150642f5765504d3974773d3d; _ga=GA1.2.1735408851.1373020219; unam=6b8dd67-13fae61e4fc-6292ed1e-43; utma=47507896.1735408851.1373020219.1422518007.1423131546.37; utmc=47507896; utmz=47507896.1409551897.29.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)"},"remoteAddress":"10.50.0.3","remotePort":55928},"response":{"statusCode":304,"responseTime":1,"contentLength":0}}

What is displayed in the browser is the same than previus post:

But, if I wait some minutes, it finally displays:

Same for Firefox and Internet explorer.

Topology in my case: elasticsearch node 1: 10.50.0.A (ES 1.4.4) elasticsearch node 2: 10.50.0.B (ES 1.4.4) kibana4 (& kibana3) node: 10.50.0.C

they all live in the same network, there is no proxies or ACLs involved. I acces to the dashboard from another network, direct routing, ports are open and in fact I contact port 5601, but just the logo loads as previusly shown. As said, kibana is running in the same machine than kibana4, and everything is fine.

In the same machine I was running Kibana4 Beta3 ok, it just stopped working today when I updated elasticsearch to 1.4.4.

rashidkpc commented 9 years ago

Does this occur without cntlm in place?

mclarke47 commented 9 years ago

I turned it off and got:

Still getting 502 bad gateway, looks like it's timing out when contacting http://localhost:5601/elasticsearch

{"@timestamp":"2015-02-20T14:07:14.989Z","level":"error","message":"connect ETIMEDOUT","node_env":"production","error":{"message":"connect ETIMEDOUT","name":"Error","stack":"Error: connect ETIMEDOUT\n at errnoException (net.js:905:11)\n at Object.afterConnect as oncomplete\n","code":"ETIMEDOUT"}} {"@timestamp":"2015-02-20T14:07:14.992Z","level":"info","message":"GET / 502 - 63011ms","node_env":"production","request":{"method":"GET","url":"/elasticsearch/","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Linux x8664; rv:31.0) Gecko/20100101 Firefox/31.0","accept":"application/json, text/plain, /_","accept-language":"en-gb,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://localhost:5601/","connection":"keep-alive","if-none-match":"W/\"33-d630f5fb\""},"remoteAddress":"127.0.0.1","remotePort":53335},"response":{"statusCode":502,"responseTime":63011,"contentLength":51}}

rashidkpc commented 9 years ago

It appears that the kibana backend can not connect to your elasticsearch server. Are you sure it is set correctly? Note that the backend connects to elasticsearch, kibana no longer connects to elasticsearch from the browser.

mclarke47 commented 9 years ago

Well Elasticsearch is definitely running and has indexes.

This is the kibana.yml

Am I missing something?

xavierromero commented 9 years ago

In my case:

[16:18:55][root@nex-es_dashboard-01 opt]# grep -v ^# kibana-4.0.0-linux-x64/config/kibana.yml port: 5601 host: "0.0.0.0" elasticsearch_url: "http://10.50.0.36:9200" elasticsearch_preserve_host: true kibana_index: ".kibana4" default_app_id: "discover" request_timeout: 300000 shard_timeout: 0 verify_ssl: true bundled_plugin_ids:

plugins/dashboard/index
plugins/discover/index
plugins/doc/index
plugins/kibana/index
plugins/markdown_vis/index
plugins/metric_vis/index
plugins/settings/index
plugins/table_vis/index
plugins/vis_types/index
plugins/visualize/index

[16:19:30][root@nex-es_dashboard-01 opt]# telnet 10.50.0.36 9200 Trying 10.50.0.36... Connected to 10.50.0.36. Escape character is '^]'.

So, connection is ok and confg is ok though. Elasticsearch logs doesn't complain about nothing.

rashidkpc commented 9 years ago

I noticed this in your logs:

"remoteAddress":"127.0.0.1","remotePort":53335

Why is the port set to 53335? Shouldn't it be 9200? Did you restart Kibana after turning off your proxy?

rashidkpc commented 9 years ago

Also, I see from your screenshot that you are running the RC1 release, you should upgrade to GA

mclarke47 commented 9 years ago

I was trying the RC1 initially, then downloaded 4 and tried with the same results.

Kibana had been restarted after turning off the proxy.

I have no idea about that port, it doesn't ring a bell and I didn't set anything to it.

I'll do some investigating around that port and try to update.

mclarke47 commented 9 years ago

I was able to get this working by turning off my proxy and stripping out all proxy environmental settings on the session used to start kibana and firefox. Thanks for the support.

pirklk commented 9 years ago

Like @mclarke47 - For Windows - I was able to recover by changing my environment variable for SET HTTP_PROXY= to clear it out... Once the proxy setting (that I need for CURL) was removed it began working properly.

kurt-o-sys commented 9 years ago

I'm having a very similar issue, but can't solve it with unsetting/stripping out all proxy env settings:

# env |grep http
# env |grep HTTP

So, nothing there.

In the browser: http://172.16.100.24:5601/ points to http://172.16.100.24:5601/#/discover?_g=(). It gives a blank screen and after a while:

Oops!
Looks like something went wrong. Refreshing may do the trick. 
Go back  or  clear your session  
Fatal Error 
 Kibana: Unknown error while connecting to Elasticsearch 

Error: Unknown error while connecting to Elasticsearch
   at Anonymous function (http://<IP-address>:5601/index.js?_b=5930:45841:11)
   at wrappedErrback (http://<IP-address>:5601/index.js?_b=5930:20882:15)
   at Anonymous function (http://<IP-address>:5601/index.js?_b=5930:21015:13)
   at $eval (http://<IP-address>:5601/index.js?_b=5930:22002:9)
   at $digest (http://<IP-address>:5601/index.js?_b=5930:21814:15)
   at $apply (http://<IP-address>:5601/index.js?_b=5930:22106:13)
   at done (http://<IP-address>:5601/index.js?_b=5930:17641:34)
   at completeRequest (http://<IP-address>:5601/index.js?_b=5930:17855:7)
   at onreadystatechange (http://<IP-address>:5601/index.js?_b=5930:17794:11)

Starting kibana seems to work fine:

 ./bin/kibana
{"@timestamp":"2015-03-06T13:05:54.536Z","level":"info","message":"Listening on  0.0.0.0:5601","node_env":"production"}

However, whenever http://172.16.100.24:5601/ is requested:

{"@timestamp":"2015-03-06T13:11:46.959Z","level":"info","message":"GET / 200 - 3ms","node_env":"production","request":{"method":"GET","url":"/","headers":{"accept":"*/*","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","dnt":"1","connection":"Keep-Alive","cache-control":"no-cache"},"remoteAddress":"<IP-address>","remotePort":60832},"response":{"statusCode":200,"responseTime":3,"contentLength":0}}
{"@timestamp":"2015-03-06T13:11:47.181Z","level":"info","message":"GET /images/initial_load.gif 200 - 19ms","node_env":"production","request":{"method":"GET","url":"/images/initial_load.gif","headers":{"accept":"*/*","referer":"http://<IP-address>/","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","dnt":"1","connection":"Keep-Alive","cache-control":"no-cache"},"remoteAddress":"<remote IP-address>","remotePort":60831},"response":{"statusCode":200,"responseTime":19,"contentLength":11934}}
{"@timestamp":"2015-03-06T13:11:47.189Z","level":"info","message":"GET /require.config.js?_b=5930 200 - 25ms","node_env":"production","request":{"method":"GET","url":"/require.config.js?_b=5930","headers":{"accept":"*/*","referer":"http://<IP-address>:5601/","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","dnt":"1","connection":"Keep-Alive","cache-control":"no-cache"},"remoteAddress":"<remote IP-address>","remotePort":60834},"response":{"statusCode":200,"responseTime":25,"contentLength":0}}
{"@timestamp":"2015-03-06T13:11:47.193Z","level":"info","message":"GET /bower_components/requirejs/require.js?_b=5930 200 - 29ms","node_env":"production","request":{"method":"GET","url":"/bower_components/requirejs/require.js?_b=5930","headers":{"accept":"*/*","referer":"http://<IP-address>:5601/","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","dnt":"1","connection":"Keep-Alive","cache-control":"no-cache"},"remoteAddress":"<remote IP-address>","remotePort":60833},"response":{"statusCode":200,"responseTime":29,"contentLength":0}}
{"@timestamp":"2015-03-06T13:11:47.194Z","level":"info","message":"GET /styles/main.css?_b=5930 200 - 32ms","node_env":"production","request":{"method":"GET","url":"/styles/main.css?_b=5930","headers":{"accept":"*/*","referer":"http://<IP-address>:5601/","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","dnt":"1","connection":"Keep-Alive","cache-control":"no-cache"},"remoteAddress":"<remote IP-address>","remotePort":60832},"response":{"statusCode":200,"responseTime":32,"contentLength":0}}
{"@timestamp":"2015-03-06T13:11:47.541Z","level":"info","message":"GET /index.js?_b=5930 304 - 0ms","node_env":"production","request":{"method":"GET","url":"/index.js?_b=5930","headers":{"accept":"*/*","referer":"http://<IP-address>:5601/","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","if-modified-since":"Wed, 04 Mar 2015 16:12:11 GMT","if-none-match":"W/\"4ec09b-1834022320\"","dnt":"1","connection":"Keep-Alive"},"remoteAddress":"<remote IP-address>","remotePort":60834},"response":{"statusCode":304,"responseTime":0,"contentLength":0}}
{"@timestamp":"2015-03-06T13:11:50.873Z","level":"info","message":"GET / 502 - 3013ms","node_env":"production","request":{"method":"GET","url":"/elasticsearch/","headers":{"referer":"http://<IP-address>:5601/#/discover?_g=()","accept":"application/json, text/plain, */*","accept-language":"nl-BE","accept-encoding":"gzip, deflate","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","host":"<IP-address>:5601","dnt":"1","connection":"Keep-Alive"},"remoteAddress":"<remote IP-address>","remotePort":60834},"response":{"statusCode":502,"responseTime":3013,"contentLength":4357}}
{"@timestamp":"2015-03-06T13:11:50.920Z","level":"info","message":"GET /bower_components/font-awesome/fonts/fontawesome-webfont.eot? 304 - 0ms","node_env":"production","request":{"method":"GET","url":"/bower_components/font-awesome/fonts/fontawesome-webfont.eot?","headers":{"accept":"*/*","referer":"http://<IP-address>:5601/","accept-language":"nl-BE","user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)","origin":"http://<IP-address>:5601","accept-encoding":"gzip, deflate","host":"<IP-address>:5601","if-modified-since":"Wed, 04 Mar 2015 16:12:11 GMT","if-none-match":"W/\"dac6-1834022320\"","dnt":"1","connection":"Keep-Alive"},"remoteAddress":"<remote IP-address>","remotePort":60834},"response":{"statusCode":304,"responseTime":0,"contentLength":0}}

checking health: http://172.16.100.24:9200/_cluster/health:

{"cluster_name":" <some custom name>","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":18,"active_shards":18,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":18}

I'm using a fresh install of elasticsearch (1.4.4) and kibana (4.0.1) without any weird config. Both are running on the same machine ().

conf/kibana.yml:

port: 5601
host: "0.0.0.0"
elasticsearch_url: "http://localhost:9200"
elasticsearch_preserve_host: true
kibana_index: ".kibana"
[...]

elasticsearch.yml

cluster.name: <some custom name>

(everything else is commented in elasticsearch.yml)

I'm behind a company proxy (which has given me quite some troubles). However, on the same machine, it shouldn't be an issue. So far, playing with the proxy settings didn't help me out (setting them, unsetting them, ...).

# uname -snmprio
Linux kbfo-bru-024 2.6.18-274.el5xen x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)

xavierromero commented 9 years ago

My case wasn't related to proxies neither. In my case it just didn't work and since I got no answers/suggestions (plus the ticket was closed), I just keep using Kibana 3. I hope newer versions will work.. !

Best regards, Xavier Romero.

mclarke47 commented 9 years ago

@qsys have you tried binding kibana to 172.16.100.24 or localhost in the kibana.yml?

MikeGoldsmith commented 9 years ago

I too was having issues connecting to my ES cluster via Kibana 4 and it was down to the index I had created not containing any items. After adding a dummy item, Kibana could successfully bind to my index via the index-pattern.

My setup has both ES & Kibana on the same box.

kurt-o-sys commented 9 years ago

@mclarke47 yes, I did :)

Anyway, it does run with unsetting http_proxy. However, I had multiple sessions running. I had to unset http_proxy on kibana and elasticsearch. Problem solved.

pkyc commented 9 years ago

Me too. Same error and work fine after remove all proxy in environment! Looks like a bugs...

robbwagoner commented 9 years ago

I am not behind a proxy, and I am having the same problem, post-upgrade from Kibana v4.0.0-beta3 to v4.0.1 and Elasticsearch 1.4.2 to 1.4.4, with Kibana and Elasticsearch on the same host.

kibana.log

{"@timestamp":"2015-03-22T23:33:15.102Z","level":"info","message":"POST /.kibana/__kibanaQueryValidator/_validate/query?explain=true&ignore_unavailable=true 200 - 4ms","node_env":"production","request":{"method":"POST","url":"/elasticsearch/.kibana/__kibanaQueryValidator/_validate/query?explain=true&ignore_unavailable=true","headers":{"host":"ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:5601","connection":"keep-alive","content-length":"79","accept":"application/json, text/plain, */*","origin":"http://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36","content-type":"application/json;charset=UTF-8","referer":"http://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:5601/","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8"},"remoteAddress":"67.131.182.97","remotePort":30886},"response":{"statusCode":200,"responseTime":4,"contentLength":221}}
{"@timestamp":"2015-03-22T23:33:15.187Z","level":"error","message":"ETIMEDOUT","node_env":"production","error":{"message":"ETIMEDOUT","name":"Error","stack":"Error: ETIMEDOUT\n  at [object Object]._onTimeout (/srv/kibana-4.0.1-linux-x64/src/node_modules/request/request.js:909:15)\n  at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)\n","code":"ETIMEDOUT"}}
{"@timestamp":"2015-03-22T23:33:15.188Z","level":"info","message":"POST /_msearch?timeout=30000&ignore_unavailable=true&preference=1427067192623 502 - 64ms","node_env":"production","request":{"method":"POST","url":"/elasticsearch/_msearch?timeout=30000&ignore_unavailable=true&preference=1427067192623","headers":{"host":"ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:5601","connection":"keep-alive","content-length":"665","accept":"application/json, text/plain, */*","origin":"http://ec2-54-158-182-185.compute-1.amazonaws.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36","content-type":"application/json;charset=UTF-8","referer":"http://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:5601/","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8"},"remoteAddress":"YY.YY.YY.YY","remotePort":51785},"response":{"statusCode":502,"responseTime":64,"contentLength":43}}
{"@timestamp":"2015-03-22T23:33:15.578Z","level":"info","message":"GET /styles/theme/elk.ico 304 - 0ms","node_env":"production","request":{"method":"GET","url":"/styles/theme/elk.ico","headers":{"host":"ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:5601","connection":"keep-alive","accept":"*/*","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","if-none-match":"W/\"47e-1834022320\"","if-modified-since":"Wed, 04 Mar 2015 16:12:11 GMT"},"remoteAddress":"YY.YY.YY.YY","remotePort":30886},"response":{"statusCode":304,"responseTime":0,"contentLength":0}}

request_time__nginx__-_visualize_-_kibana_4

kibana.yml I've attempted to bind Kibana to both all interfaces (0.0.0.0) and only the eth0 (10.102.151.13)

# Kibana is served by a back end server. This controls which port to use.
port: 5601

# The host to bind the server to.
host: "10.102.151.13"
#host: "0.0.0.0"
#host: "127.0.0.1"

# The Elasticsearch instance to use for all your queries.
elasticsearch_url: "http://localhost:9200"

elasticsearch_preserve_host: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizations
# and dashboards. It will create a new index if it doesn't already exist.
kibana_index: ".kibana"

# The default application to load.
default_app_id: "discover"

# Time in seconds to wait for responses from the back end or elasticsearch.
# Note this should always be higher than "shard_timeout".
# This must be > 0
request_timeout: 60

# Time in milliseconds for Elasticsearch to wait for responses from shards.
# Note this should always be lower than "request_timeout".
# Set to 0 to disable (not recommended).
shard_timeout: 30000

# Set to false to have a complete disregard for the validity of the SSL
# certificate.
verify_ssl: true

# Plugins that are included in the build, and no longer found in the plugins/ folder
bundled_plugin_ids:
 - plugins/dashboard/index
 - plugins/discover/index
 - plugins/doc/index
 - plugins/kibana/index
 - plugins/metric_vis/index
 - plugins/settings/index
 - plugins/table_vis/index
 - plugins/vis_types/index
 - plugins/visualize/index

Kibana (Node pid 5150) is connected to Elasticsearch (Java pid 4361) port 9200:

$ sudo netstat -tpan |grep 9200
tcp        0      0 127.0.0.1:39271         127.0.0.1:9200          ESTABLISHED 5150/node
tcp6       0      0 :::9200                 :::*                    LISTEN      4361/java
tcp6       0      0 127.0.0.1:9200          127.0.0.1:39271         ESTABLISHED 4361/java

robbwagoner commented 9 years ago

My variant of this issue is likely caused by the change for request_timeout from Kibana 4.0.0-beta3 to 4.0.1. Its unit was seconds, but is now milliseconds. I'll confirm once tested and verified. See Issue 2813

xavierromero commented 9 years ago

Just to inform, I got this fixed with Kibana 4.0.1 !!! :-)

kuldeeparyadotcom commented 9 years ago

For me, it was because of http_proxy and https_proxy environment variable. I unset those and then kibana4 could connect with elasticsearch1.4.4.

bukkasamudram commented 9 years ago

I am also facing same issue. Can you please let us know how to unset http_proxy and https_proxy environment variables.

mclarke47 commented 9 years ago

Call the following commands in the session you use to start kibana:

UNSET http_proxy
UNSET https_proxy
UNSET HTTP_PROXY
UNSET HTTPS_PROXY

bukkasamudram commented 9 years ago

Hi, I did same in my linux box session, but same result. "Kibana: Unknown error while connecting to Elasticsearch". Can you please help us to resolve this issue.

a-shlash commented 9 years ago

I've the same problem, and work fine after stop proxy for localhost ubuntu@pc:kibana-4.0.2-linux-x64$ export no_proxy="localhost" hope that helps

bukkasamudram commented 9 years ago

Hi, we are facing index status health problems. For all the indexes showing health as red. can you please help us in resolving this.

http://localhost:9200/_cat/indices?v health status index
red open gnwvvo1.html 5 1
yellow open logstash-ng-qa-2015-05-22 5 1
yellow open logstash-golf-qa-2015.05.27 5 1
red open logstash-golf-qa-2015.06.02 5 1
red open logstash-golf-qa-2015.05.31 5 1
red open logstash-golf-qa-2015.05.29 5 1
yellow open logstash-golf-qa-2015.05.30 5 1
red open phppath 5 1
red open logstash-ng-qa-2015-05-23 5 1
red open logstash-golf-qa-2015.06.01 5 1
yellow open spipe 5 1
yellow open oogsen1.html 5 1
yellow open eswong1.html 5 1
yellow open logstash-ng-qa-2015-05-24 5 1
red open logstash-ng-qa-2015.05.31 5 1
red open logstash-golf-qa-2015.05.23 5 1
red open logstash-ng-qa-2015.05.27 5 1
red open logstash-ng-qa-2015-05-25 5 1
yellow open logstash-golf-qa-2015.05.25 5 1
red open logstash-ng-qa-2015.05.28 5 1
red open logstash-ng-qa-2015.05.30 5 1
red open logstash-ng-qa-2015.05.29 5 1
yellow open logstash-golf-qa-2015.05.24 5 1
red open ngevpp1.html 5 1
yellow open webui 5 1
yellow open perl 5 1
red open logstash-ng-qa-2015.06.01 5 1
yellow open logstash-ng-qa-2015.05.26 5 1
red open logstash-golf-qa-2015.05.26 5 1
red open logstash-ng-qa-2015-05-21 5 1
red open logstash-golf-qa-2015.05.28 5 1
yellow open logstash-golf-qa-2015.05.21 5 1
yellow open .kibana 1 1
yellow open logstash-golf-qa-2015.05.22 5 1
red open logstash-ng-qa-2015.06.02 5 1

rothwm commented 9 years ago

I'm having very similar problems...

I'm new and trying to experiment with ELK.

I have twitter feeds in ES from Logstash, they are stored and I can retrieve from the terminal. I can run Kibana 4.0.2 and can create the time index from settings but when I go to visualise and select the time histogram, selecting the @timespan noting is being displayed, no records found. All very frustrating. Any advice??

sand0id commented 9 years ago

I had a similar problem when running a cluster with a firewall between data and non-data nodes. The cluster was quite unstable, with nodes often leaving and joining. Kibana would fail to load after requesting /elasticsearch/nodes?=[timestamp]. Simplifying the cluster seems to have sorted it out.

elastic / kibana

Kibana 4 can't connect to local Elasticsearch #3112