DelaydJob SQL request are very slow

tobi / delayed_job

Database backed asynchronous priority queue -- Extracted from Shopify

http://tobi.github.com/delayed_job

MIT License

2.15k stars 1.25k forks source link

DelaydJob SQL request are very slow #24

Open mikhailov opened 14 years ago

mikhailov commented 14 years ago

NewRelic RPM show me the DelaydJob SQL request as very slow. MySQL Slow Query Log does the same. What exactly column should be indexed? single or multiple columns indexes? Does anybody do an analysis?

mikhailov commented 14 years ago

I have found an interested article, Tobi, please take a look, can it help? http://www.elevatedcode.com/articles/2009/11/04/speeding-up-delayed-job/

jsuchal commented 14 years ago

You need this. http://github.com/collectiveidea/delayed_job/commit/c743446a31fe5683544b78e9bf86dec371923b38

Depending on which database you use there are different solutions:

with MySQL you should reverse priority order and create a composite[:priority, :run_at],
with Postgres you should create a composite index with different order on both columns priority DESC, run_at ASC.

Hope this helps.

BTW I am using the former "method" with thousands of jobs in table with no problem.

mikhailov commented 14 years ago

Great thanks! But I'm using of gem instead of plugin. Is it possible to pull request to official version?

jsuchal commented 14 years ago

You can also use it as a gem. Just look at README http://github.com/collectiveidea/delayed_job#readme