search

ros-infrastructure / rosindex

The source code for generating index.ros.org
https://index.ros.org
GNU General Public License v3.0
12 stars 24 forks source link

Slow search on rosindex #176

Open mortenfyhn opened 4 years ago

mortenfyhn commented 4 years ago

I hope this is the right spot for this kind of feedback.

When I search on index.ros.org, it takes around ten seconds to get the search results. The old index simply listed everything on one page, so I could use my browser's search function, which is immediate. How about adding a "show all" button on the main index page to get that possibility back? The default can still be 50 elements per page.

hidmic commented 4 years ago

@mortenfyhn does it always take ~10s? Or just the first time it loads? Most of the time used to and should be spent pulling the lunr.js-based index, not searching. Ten seconds is way too much even for that anyways.

mortenfyhn commented 4 years ago

Seems to be first time only, after that it takes roughly 2.5 s. I'm using a Latitude 7490 laptop on a 100 Mbps line, so that should be okay.

hidmic commented 4 years ago

Yeah, it's the time it takes to pull the lunr.js index. We can further split it in chunks, but I don't how much that'd improve things. @tfoote any ideas?

tfoote commented 4 years ago

If anything I think more chunking will slow things down. It looks like we're actually spending more time waiting for chunks than downloading them, and they're pretty small at the moment. Can we parallelize the downloads? Or can we background load the chunks while the searchbox is on the screen or at least as soon as the user starts typing in the search box?

Here's the network timeline I see getting the index files. That's only about half the index files. The rest are even smaller and load sequentially quicker. Screenshot from 2019-09-16 16-08-19

tfoote commented 4 years ago

We could also try out requestIdleCallback to background load it.

ros-discourse commented 5 months ago

This issue has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/growing-issue-with-ros-documentation/36075/44

mortenfyhn commented 5 months ago

I'll update:

First of all: This is faster now. I'm not sure if it's faster devices and internet connection or the website is faster or both.

Within a browser tab, it seems to cache perfectly so subsequent searches are very fast. If I open a new tab, I get the loading again. But the loading is much more reasonable than what I experienced back in 2019.

Might be worth checking on a more modest computer, and with a browser that's guaranteed to not already have stuff cached.

https://github.com/ros-infrastructure/rosindex/assets/6639836/88a05369-104e-4f50-9559-9cddfc8e681a