search

pat / thinking-sphinx

Sphinx/Manticore plugin for ActiveRecord/Rails
http://freelancing-gods.com/thinking-sphinx
MIT License
1.63k stars 468 forks source link

including thinking sphinx in gemfile significantly slows down the development env in rails 3.1 #282

Closed josh-m-sharpe closed 13 years ago

josh-m-sharpe commented 13 years ago

I have a mid-sized app, ~24 gems including:

gem 'thinking-sphinx', '2.0.7', :require => 'thinking_sphinx' gem 'ts-datetime-delta', '1.0.2', :require => 'thinking_sphinx/deltas/datetime_delta'

I've recently upgraded it to 3.1, and assets (css, js, images) compilation/serving is noticeably slow. At first, I thought it was rails doing this, but after process of elimination have narrowed it down to simply including TS in my gemfile.

Commenting out those two lines, as well as my define_index blocks, re-bundling and restarting my dev server makes for a notable increase in speed when moving around my app.

Requests that previously took 6-7 seconds, now take 1-2..... all for a response that was generated in <500ms.

Note that the pages I'm viewing do not hit TS at all - so TS must be hooked into the controller/views somehow?? Why would it be doing that to begin with?

TS may not be completely to blame, but something fishy is definitely going on. I just wanted to see if anyone else has experienced this.

pat commented 13 years ago

I just added something to the docs about this - the issue is probably Thinking Sphinx loading all of your models on every request (in development mode). Any request that deals with a model that is indexed (whether or not there's a search call) will hit this problem, I'm pretty sure.

There is a workaround, though: http://freelancing-god.github.com/ts/en/common_issues.html#slow-page-requests

josh-m-sharpe commented 13 years ago

You SO Sexy. But.... your gist link is broken....

pat commented 13 years ago

Ergh, got the path slightly wrong. Will fix now, here's the right link: http://freelancing-god.github.com/ts/en/advanced_config.html#indexed-models

josh-m-sharpe commented 13 years ago

Ok, got it in place. Can't say I notice that it's much faster.

FWIW, 56 models here, so dropping it to 3 index models, if that were the case, probably should've had an impact(?)

Maybe there's something about these asset requests that TS can pick up on before doing its thing?

Would you mind pointing out where this scan happens?

pat commented 13 years ago

Yeah, I'm surprised things haven't improved much.

Scan happens in ThinkingSphinx::Context.

josh-m-sharpe commented 13 years ago

You said that each request that hits a model would have this problem, however:

Started GET "/assets/orders.css" for 127.0.0.1 at 2011-08-31 20:47:03 -0400 Served asset /orders.css - 304 Not Modified (4ms)

doesn't hit a model, but it's still slow.

pat commented 13 years ago

Oh... that's not cool. I have a feeling this isn't the issue I first thought it was.

pat commented 13 years ago

As discussed elsewhere, we figured it was the I18n calls that caused the slowdown. I just ran through tests on Rails 3 and 3.1. Doesn't look like the load order problem for translations occurs any more, so have removed those two lines, things should be faster now :)