#<Errno::ECONNREFUSED: Connection refused - connect(2)> on boot

[davidhorsak]

Hi,

I am writting about ElasticSearch in my bachelor thesis comparing it with Apache Solr and PostgreSQL.

Rails app runs on my Mac and search engines run in VMs in VirtualBox. All are running on Ubuntu 10.04 LTS. The error appears when boot to any environment, start console, start server, run rake task, and it looks like this:

$ bundle exec rails c
Skipping index creation, cannot connect to ElasticSearch
(The original exception was: #<Errno::ECONNREFUSED: Connection refused - connect(2)>)
Loading production environment (Rails 3.2.3)
1.9.3p0 :001 > ...

I created initializer that configures Tire with the IP of machine where ES runs.

Tire.configure do 
  url "http://192.168.1.138:9200"
end

I tried connecting to ES using CURL and it went smoothly as well as with HTTParty through Rails console:

$ curl http://192.168.1.138:9200
{
  "ok" : true,
  "status" : 200,
  "name" : "Magilla",
  "version" : {
    "number" : "0.19.0",
    "snapshot_build" : false
  },
  "tagline" : "You Know, for Search"
}

I did even tried searching and it worked!

 User.tire.search do
   query { string "davetsunami" }
 end
 => #<Tire::Results::Collection:0x007f9455cc9ea8 @response=...> 

It looks like Tire is not able to connect to ES on startup but then it works perfectly. I saw #180 and configures ES with IP address of its machines but it didn't help.

Please have a look at this and tell me if its my local problem or if it is something bigger. Thanks!

[karmi]

Hard to say what's your problem exactly... If ES is running on http://192.168.1.138:9200 when you start the console, Tire should be able to connect to it.

[dennisreimann]

We are experiencing similar errors, that's why I made a pull request to have a better debugging output for connection errors.

[davidhorsak]

It gets connected, but during the boot it throws the mentioned error.

Thanks @dbloete, I tried to use your branch and now I get

Skipping index creation, cannot connect to ElasticSearch
(The original exception was: #<Errno::ECONNREFUSED: Unable to connect to ElasticSearch on http://localhost:9200/users>)
Loading development environment (Rails 3.2.3)
# ...

I went throught the code and in console tried both ways of checking if the index exists. I used the curl way and the rest client way found in tire / lib / tire / index.rb and both were 200 success.

It really stopped having any sense, then I tried putting binding.pry before Tire configuration and the curious thing was that it threw the connection error before Tire even got configured!

$ bundle exec rails c
Skipping index creation, cannot connect to ElasticSearch
(The original exception was: #<Errno::ECONNREFUSED: Unable to connect to ElasticSearch on http://localhost:9200/users>)

From: /Users/davetsunami/Projects/Artvasion/config/initializers/elasticsearch.rb @ line 1:

 => 1: binding.pry
    2: 
    3: Tire.configure do 
    4:   url "http://192.168.1.138:9200" #APP_CONFIG[:elasticsearch]["host"]
    5: end
    6: 

[1] pry(main)> Tire::Configuration.url
=> "http://localhost:9200"
[2] pry(main)> exit
Loading development environment (Rails 3.2.3)
# ...

[karmi]

The problem you're having still does not make any sense to me, sadly.

Configuring Tire in an initializer is perfectly fine, and works. Maybe try to generate the example application and point it to your virtual machine -- hard to say what's going on in your app otherwise.

[davidhorsak]

The example app works without any connection problems.I tried connecting it with the provided ElasticSearch in vendor/ and my remote ElasticSearch server and both worked with Tire perfectly. I'll do further investigations and report back.

[davidhorsak]

Got it. There seems to be an interference between Tire and Devise. Tire initializer is called elasticsearch.rb and Devise initializer is called devise.rb. When I placed Tire configuration before Devise configuration, it started OK with no exception messages. When I placed it back after Devise configuration, I got the thousand times mentioned connection error.

This is how my Devise config initializer looks:

Devise.setup do |config|

  # ==> ORM configuration
  # Load and configure the ORM. Supports :active_record (default) and
  # :mongoid (bson_ext recommended) by default. Other ORMs may be
  # available as additional gems.
  require 'devise/orm/active_record'

  config.mailer_sender = "no-reply@artvasion.com"
  config.case_insensitive_keys = [ :email ]
  config.strip_whitespace_keys = [ :email ]
  config.http_authenticatable_on_xhr = false
  config.paranoid = true
  config.stretches = APP_CONFIG[:stretches]
  config.pepper = APP_CONFIG[:pepper]
  config.remember_for = 2.weeks
  config.extend_remember_period = true
  config.use_salt_as_remember_token = true
  config.rememberable_options = {:secure => true}
  config.password_length = User::PASSWORD_MIN_LENGTH..User::PASSWORD_MAX_LENGTH
  config.lock_strategy = :failed_attempts
  config.unlock_strategy = :none
  config.maximum_attempts = 1/0.0
  config.reset_password_within = 1.day
  config.sign_out_all_scopes = true
  config.sign_out_via = :delete
end

There's definitely something in that require 'devise/orm/active_record'

[xinming]

We are having the exact same problem, and despite how little sense it makes, the fix mentioned by @DaveTsunami works for us.. Although we have mongoid instead of active_record in that line.

[bashcoder]

Exact same issue here - conflict with Devise initialization. Same fix by @DaveTsunami worked for me.

[elfassy]

+1 (good catch @DaveTsunami)

[verdi327]

I am having the same issue, except I am not using Devise. I hava no auth system in my app and I can imagine any other initializer I have that would be interfering. My app is launched on Heroku at http://dynamiteurbanite.herokuapp.com/ and I am unable to get ES working consistently. I am using the Bonsai add on provided by Heroku. I run thru the practice examples provided on the add on page for Bonsai and everything responds how it should. For example I can run this command

curl -XPOST http://index.bonsai.io/q3s8a05750388n7sa9wo/article -d '{"title": "Hello, Bonsai."}'

and I receive a JSON response telling me everything is working ok

{ "ok": true, "_index": "q3s8a05750388n7sa9wo", "_type": "article", "_id": "7yYC-inLT7-eFkAdCAUv2A", "_version": 1 }

But, when I go to enter a search param the app breaks. Checking my heroku logs I see that I get this error everytime...

Started GET "/cities?utf8=%E2%9C%93&query=georgia" for 69.250.146.115 at 2012-07-04 01:12:53 +0000 2012-07-04T01:12:53+00:00 app[web.1]: Processing by CitiesController#index as HTML 2012-07-04T01:12:53+00:00 app[web.1]: Parameters: {"utf8"=>"✓", "query"=>"georgia"} 2012-07-04T01:12:56+00:00 app[web.1]: Completed 500 Internal Server Error in 3001ms 2012-07-04T01:12:56+00:00 app[web.1]: Errno::ECONNREFUSED (Connection refused - connect(2)):

The weirdest part is that it was working at one point - I don't know how it fixed itself - but then when I went to use the app later that night (no code changes) I am back receiving this same error.

Same guidance would be awesome!!!!

[wflanagan]

Same here, no devise. But, I have this error. Anyone have any thoughts on how to fix?

[elfassy]

Are you sure you have the correct ip and port?

Tire.configure do 
  url "http://192.168.1.138:9200"
end

[karmi]

I am using the Bonsai add on provided by Heroku

I'm not sure what's the current status of Bonsai and Tire -- there has been numerous bugs being reported.

[wflanagan]

Yes, I tried it with that. The port helps (9200).

I saw the bugs, so I swung it off of Bonsai and onto my own EC2 instance. I'm not having that problem anymore.. but I am having another. I have posted a question on it (with configs) to http://stackoverflow.com/questions/12041831/elasticsearch-tire-keywords-right-way-to-match-or-for-a-keyword-list

Thanks for the feedback on this issue.

[karmi]

@wflanagan terms is what you need, and also boolean queries, see https://github.com/karmi/tire/blob/master/test/integration/boolean_queries_test.rb

[nz]

CC'ing myself on some issues mentioning Bonsai…

An ECONNREFUSED is a pretty vague exception to debug without more context on the stack trace and other circumstances. I like the idea behind @dennisreimann's PR #323 to help with that kind of thing…

If an ECONNREFUSED is ever being generated by someone using Bonsai, I'd love to hear about that at suport@onemorecloud.com, and I can be collabed in to Heroku apps to help debug errors like that.

[karmi]

All: Please direct all questions about Bonsai to their support channels.

Strange stuff with the Devise and all... Closing the issue for now...

[kristopher]

This is pretty old but I just ran into it so I thought I would leave an explanation for future reference. I'm not using devise but I believe this will happen if any initializer causes a model to load that includes a tire index. The model loads which causes tire to load the mapping and in doing so it tries to create the index but at the time the ENV['ELASTICSEARCH_URL'] hasn't been set yet by the tire/elasticsearch initializer so tire defaults to localhost and bam connection refused.

[karmi]

@kristopher That sounds like a good explanation and debugging! I'm afraid there's no way around it than proper documentation?

[kristopher]

Documentation sounds like the right approach to me. The solution is simple once you know what the issue is.

[nz]

Documentation should work pretty well for this. Almost every production ECONNREFUSED error I've seen comes down to a misconfigured client incorrectly trying to connect to its development default of localhost.

Other ideas in addition to documentation…

Printing the full URL on ECONNREFUSED, or at least including the host, should go a long way toward helping people troubleshoot this kind of issue on their own. There's already a pull request, #323, that would work well for this.

Based on @kristopher's explanation, it sounds like there's a request executed against Elasticsearch while some class is loaded for other purposes. Is that strictly necessary? Can that request be deferred?

[kristopher]

Printing the URL sounds like a fine idea more info the better.

In addition you could get away from defaulting to localhost and throw an exception if the url is not set in the env or through the configuration interface. Seems like that could give the user a better idea of what the issue is and point them in the right direction when they go to debug. I'm not sure it's worth going for configuration over convention in this place but it's an idea.

[karmi]

Printing the URL sounds like a fine idea more info the better.

Agreed.

(...) you could get away from defaulting to localhost and throw an exception (...)

...and breaking the experience for everyone, because you have to set localhost:9200 all the time. You just can't prevent people to shoot them in the foot by not paying enough attention. More importantly, I don't remember all the issues raised in this ticket, but the original issue is about the informative mesage Skipping index creation.

[kristopher]

I think "breaking the experience" and "shoot them in the foot" is a bit over dramatic but I certainly understand where you are coming from. Just throwing an idea out there.

[iwarshak]

In the interest of completeness, moving the bonsai config into environment.rb did the trick

ENV['ELASTICSEARCH_URL'] = ENV['BONSAI_URL'] 

# Initialize the rails application
App::Application.initialize!

[karmi]

@iwarshak Thanks! By the way, I think the ELASTICSEARCH_URL and BONSAI_URL should be preferably set with heroku config?

[karmi]

@nz All this issues would go away if you'd be able to require "tire/heroku" or require "tire/bonsai" and all these checks like _Is ENV['ELASTICSEARCHURL'] properly set?, _Is BONSAIURL set?, Can we talk to the index?, etc.

[iwarshak]

@karmi The heroku/bonsai docs say to set the ELASTICSEARCH_URL variable in an initializer. Perhaps because the BONSAI_URL could be changed by them during a migration or something

[nz]

@karmi re: require "tire/bonsai" — good idea, I had forgotten we'd talked about that. Asking people to manually create initializers can be a bit fiddly. Let me ping you on IM in a bit to figure out a good way to package that up. Maybe a tire-bonsai gem which I maintain separately?

[karmi]

Yeah, please ping me. In the future, I envision the client to be much more modular like this anyway, so it would all play nicely.