internetarchive / openlibrary

One webpage for every book ever published!
https://openlibrary.org
GNU Affero General Public License v3.0
5.22k stars 1.37k forks source link

Changes not showing up in autocomplete & search contexts #4690

Open tfmorris opened 3 years ago

tfmorris commented 3 years ago

Search indexing of changes used to be quite slow, but reasonably reliable. Now it seems to be somewhat faster, if it happens, but increasingly unreliable. Search is the glue that holds the system together because it's used not only to power user searches, but also to dynamically construct many different types of pages. It must become a priority and be made reliable.

Evidence / Screenshot (if possible)

Here are some examples. I created this author record 12 days ago: https://openlibrary.org/authors/OL9105182A and it still is inaccessible both by name and ID in the work edit form autocomplete Screen Shot 2021-02-28 at 9 04 27 PM Additionally the work that edited to create that record: https://openlibrary.org/works/OL20688435W Doesn't show up on the new author page and still shows up on the old author page: https://openlibrary.org/authors/OL3966915A

This author record: https://openlibrary.org/authors/OL768915A still shows works that were edited to correct the author two days ago:

The results for this author search: https://openlibrary.org/search/authors?q=jenny+morris&mode=everything shows 45 books for the top author, Screen Shot 2021-02-28 at 9 21 46 PM but the author page shows 40: https://openlibrary.org/authors/OL556098A and claims there are three pages of results, even though the third page is empty: https://openlibrary.org/authors/OL556098A/Jenny_Morris?page=3

Screen Shot 2021-02-28 at 9 24 26 PM

This author is missing newly edited metallurgy works from a couple of days ago: https://openlibrary.org/authors/OL6457565A which are still listed under this author: https://openlibrary.org/authors/OL1716362A

etc, etc

This happens on an ongoing basis and is a waste of users' volunteered efforts.

Relevant url?

See above.

Steps to Reproduce

  1. Go to ... description above
  2. Do ...

Details

Proposal & Constraints

Related files

Stakeholders

lephemere commented 3 years ago

This PR #5054 should resolve the issue about search giving a last empty page. Results page count calculation was giving +1 empty page for search with multiple of 20 results.