kaminari performance terrible with large number of pages

[MrDys]

CODEBASE-350: See: https://github.com/amatsuda/kaminari/issues/137

I am not completely sure if I diagnosed the problem right. What I do know is that my app was performing terribly (20s response times on queries with many hits), and the majority of the problem went away when I eliminated kaminari pagination.

If this really is a general problem wtih kaminari pagination on a large hit count (some confirmation from others would be nice), then we need to either patch kaminari to not have this problem, or stop using kaminari in Blacklight. It is an expected BL use case to support very large total-num-hits query results.

[MrDys]

Original reporter: jrochkind

[MrDys]

cbeer: https://github.com/projectblacklight/blacklight/commit/0e64d634fbfe7e8a5bab7e655d261aa34433b75d

Should still monitor Kaminari #137 and #154 to see if the fix gets included upstream.

[MrDys]

cbeer: Local applications that overrode the blacklight kaminari theme can use the patch by updating their theme's _pagination partial to change the #each_page call to #each_relevant_page

[MrDys]

jrochkind: Sorry, now that I tried it, your snippet is roughly the UI I'd want, thanks for providing that! I completely don't understand how it works, but okay then. I wonder if we should put this in BL?

On 7/14/2011 9:33 AM, Chris Beer (JIRA) wrote:

 [ http://jira.projectblacklight.org/jira/browse/CODEBASE-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=11843#action_11843 ]

Chris Beer commented on CODEBASE-350:

If and until Kaminari addresses this in core, local implementations with very large page counts can and should feel free to override ./app/views/kaminari/blacklight/_paginator.html.erb to be a little smarter about rendering pages. One approach, albeit ugly, is https://gist.github.com/b7590aed87ea7671da63

kaminari performance terrible with large number of pages

             Key: CODEBASE-350
             URL: http://jira.projectblacklight.org/jira/browse/CODEBASE-350
         Project: Blacklight Plugin
      Issue Type: Bug
        Reporter: Jonathan Rochkind
        Assignee: Chris Beer
         Fix For: 3.1

See: https://github.com/amatsuda/kaminari/issues/137 I am not completely sure if I diagnosed the problem right. What I do know is that my app was performing terribly (20s response times on queries with many hits), and the majority of the problem went away when I eliminated kaminari pagination. If this really is a general problem wtih kaminari pagination on a large hit count (some confirmation from others would be nice), then we need to either patch kaminari to not have this problem, or stop using kaminari in Blacklight. It is an expected BL use case to support very large total-num-hits query results.

[MrDys]

cbeer: <%# The container tag

• available local variables current_page: a page object for the currently displayed page num_pages: total number of pages per_page: number of items to fetch per page remote: data-remote paginator: the paginator that renders the pagination tags inside -%> <%= paginator.render do -%>
  <%= prev_page_tag %> <%= next_page_tag %>

  <% options = @window_options.merge(@options) %> <% [1.upto(options[:left]).to_a, (options[:num_pages] - options[:right] + 1).upto(options[:num_pages]).to_a, (options[:current_page] - options[:window] - 1).upto(options[:current_page] + options[:window] + 1).to_a].flatten.uniq.sort.reject { |x| x < 1 or x > options[:num_pages] }.map { |i| Kaminari::Helpers::Paginator::PageProxy.new(options, i, @last) }.each do |page| -%> <% if page.left_outer? || page.right_outer? || page.inside_window? -%> <%= page_tag page %> <% elsif !page.was_truncated? -%> <%= gap_tag %> <% end -%> <% end -%>

  <% end -%>

[MrDys]

cbeer: If and until Kaminari addresses this in core, local implementations with very large page counts can and should feel free to override ./app/views/kaminari/blacklight/_paginator.html.erb to be a little smarter about rendering pages. One approach, albeit ugly, is https://gist.github.com/b7590aed87ea7671da63

[MrDys]

jrochkind: That snippet is, I think, not an acceptable UI for us here, so I'll have to come up with an alternative locally, but it'll probably be too hacky and customized for our needs to commit to BL. (I'm pretty overwhelmed right now).

I do think that we should not be comfortable shipping a Blacklight that exhibits this problem -- if Kaminari doesn't address it, we still have to, either by not using kaminari out of the box or patching kaminari. I don't think it's acceptable long-term to say this is a known problem and the answer is everyone needs to individually local patch. I understand it's the reality short term until someone finds time to do something about it.

On 7/14/2011 9:33 AM, Chris Beer (JIRA) wrote:

 [ http://jira.projectblacklight.org/jira/browse/CODEBASE-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=11843#action_11843 ]

Chris Beer commented on CODEBASE-350:

If and until Kaminari addresses this in core, local implementations with very large page counts can and should feel free to override ./app/views/kaminari/blacklight/_paginator.html.erb to be a little smarter about rendering pages. One approach, albeit ugly, is https://gist.github.com/b7590aed87ea7671da63

kaminari performance terrible with large number of pages

             Key: CODEBASE-350
             URL: http://jira.projectblacklight.org/jira/browse/CODEBASE-350
         Project: Blacklight Plugin
      Issue Type: Bug
        Reporter: Jonathan Rochkind
        Assignee: Chris Beer
         Fix For: 3.1

See: https://github.com/amatsuda/kaminari/issues/137 I am not completely sure if I diagnosed the problem right. What I do know is that my app was performing terribly (20s response times on queries with many hits), and the majority of the problem went away when I eliminated kaminari pagination. If this really is a general problem wtih kaminari pagination on a large hit count (some confirmation from others would be nice), then we need to either patch kaminari to not have this problem, or stop using kaminari in Blacklight. It is an expected BL use case to support very large total-num-hits query results.