wagtail / wagtail-autocomplete

An Autocomplete edit handler for selecting Pages, Snippets, and more.
https://wagtail-autocomplete.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
120 stars 54 forks source link

Bug with search suggestions #171

Open jsma opened 9 months ago

jsma commented 9 months ago

We've noticed an odd issue with how React is handling search suggestions (disclaimer, I'm unfamiliar with React and there isn't a dist.js.map included so my debugging abilities are somewhat limited).

For example, we have a field for US Counties. I can see in my browser's inspect tool that on initial page load, it automatically loads in the first 100 results but keeps them hidden until I click into the search field.

Screenshot 2023-12-08 at 12 12 41 PM

Next, I attempt to search for "Bucks County, PA". First I type "b" which shows results that contain "b":

Screenshot 2023-12-08 at 12 13 39 PM

As I type the rest of "buck", I briefly see it narrow things down to the only two results that should match:

Screenshot 2023-12-08 at 12 14 09 PM

But then it goes haywire and reverts to showing all the results that it fetched when I typed "bu":

Screenshot 2023-12-08 at 12 13 56 PM

I've noticed that this happens when I type quickly but if I type slowly, one character at a time, and wait for the suggestion results to display for each character typed, then the correct suggestions are shown and are sticky, it doesn't bounce back to some previous result set. If I type faster than it can update the display of results, it still fetches the results for each step of the query but gets very confused about which result set it should display for the current input value.

In the above example, the only reason it reverted to the "bu" results is that I paused after typing the "b" to get a screen shot for this issue. If I reload the page and quickly type the full word "buck", it will revert to the results for the letter "b".

jsma commented 8 months ago

The actual issue here is a race condition where the "buck" results were returned before the "b" or "bu" results. There is nothing in the JSON response to indicate to the front end which query the results are for, so when the Promise finally resolves for 100 slower results, it erases the two correct results which were returned first.

I believe the solution to this bug is to include the search query with the results so that the front end can make sure it's displaying the correct results if and only if the results are for the value of the current input instead of letting the most recent results win.

In our case, we were querying against a table with many columns of large GeoDjango data that was causing general sitewide mayhem due to the CPU/RAM demands for creating Python objects out of it. Our general solution was to create unmanaged models to only include the most commonly needed columns (e.g., ID, county name, state abbreviation) since it is otherwise non-trivial to get Wagtail or this autocomplete package to use e.g., only() or other queryset optimizations.

Now that results are returning so much more quickly, this issue is less of a problem. Nevertheless, the potential for this bug still remains due to similar database performance issues, network performance, etc. I'll post a PR shortly.

muvalika commented 5 months ago

This above issue can be fixed by: It majorly challenges the troubleshooting. As it deals with asynchronous modifications. It can be implemented by the real time filtering, handling with the data fetches and renders. It is basically a biopolaring of the debouncing and ensuring. Optimizing the API calls.