search

mozilla / mozhacks-2013

The old/classic hacks.mozilla.org WordPress theme. New theme at https://github.com/potch/hax
55 stars 25 forks source link

Hacks2010 and 2013 flush/regen rewrite_rules constantly, kills performance #39

Open superawesome opened 10 years ago

superawesome commented 10 years ago

https://github.com/mozilla/mozhacks/blob/master/themes/Hacks2013/functions.php#L542

Maybe relevant documentation: http://codex.wordpress.org/Function_Reference/add_filter http://adambrown.info/p/wp_hooks/hook/init

TL;DR: It seems like this should be done only on very select events (new post, new page, adding/removing a plugin, changing themes, etc). Instead, it appears to be happening on every single hit.

The effect is that the wp_options table in the database grows unbounded, and DB response time slowly grows. At a 2.2GB table, we were seeing over 90 second response times. This table should be very small... it's only 200-400 rows. Most such tables for blog.mozilla.org blogs are well under 1MB. The current hacks.mozilla.org table is ... 516MB, and growing rapidly. It will take only a few hours until response time suffers significantly and the table needs to be recreated (optimized, shrank) again.

The more pages/posts there are, the larger the INSERT/DELETE cycle is, and the more hits there are, the more often the cycle happens.

superawesome commented 10 years ago

Bug 1063213 is the relevant internal Mozilla bug on this issue.

craigcook commented 10 years ago

Commented on the bug, reposting here to save you a click:

It seems all that rewrite business is related to the custom views and sorting, where you can view posts as titles, summaries, or full posts, and sort by date or number of comments. All those options work on a query string but there are rewrites to make the URLs friendlier. I'm not sure why the flushing would be called repeatedly instead of only on activation (might have been a simple mistake or an artifact from testing).

I'll see if it's as simple as only calling that function on activation and if that still lets the custom views work, but a more drastic quick fix would be to just rip out all the custom view stuff entirely (I don't know if anybody even uses it; it's certainly not essential).

robnyman commented 10 years ago

Where is that custom view functionality used/available? The only way I use it is from the list of posts to look at metrics (e.g. published posts over time, posts per a specific month, number of authors as an extension of that).

craigcook commented 10 years ago

It's this bit: hacks-view-sort You can sort by date or number of comments, and view the list as titles, summaries, or full posts. Any of those options just requests the page with some parameters to do a custom loop, and all this rewriting stuff makes the URLs a bit nicer, so you get 'https://hacks.mozilla.org/articles/by/comments/as/title/' instead of a query string.

The post sorting and view switching was part of the original requirements when we designed and built Hacks back in 2009 but I have no idea if real users get any value from it. The fix in pull #40 might address the performance issues with the rewrites, but if not, we still have the option of just removing the entire feature.

robnyman commented 10 years ago

Ah, I see. If the fix doesn't address it completely, and removing the custom views might, I'm behind trying that. I have no numbers for how many people use it (would take a deep-dive into Google Analytics), but it's far from a key feature.

groovecoder commented 10 years ago

Did soemoen say deep-dive into Google Analytics? 58 page-views in the last month of traffic, so no - not a key feature. ;)

robnyman commented 10 years ago

Ha! :-) Thanks!

groovecoder commented 10 years ago

Note: we just put this into our board for this week: https://trello.com/c/8ldxCghd/154-bug-1063213-pr-40-hacks-mozilla-org-is-stuck-in-a-redirect-loop so either @darkwing or I should get to it soon.