search

trakt / api-help

Trakt API docs at https://trakt.docs.apiary.io
182 stars 7 forks source link

Search queries improvement #60

Closed kevincador closed 5 years ago

kevincador commented 5 years ago

We had a report today from a user looking for the movie ‘She’s All That’ (1999). If he's searching for ‘She’s All That’ nothing is found. If he's searching for ‘All That’ the first result in the list is the right movie spelled exactly as he tried the first time. He said he found this same issue when looking up for other films with an apostrophe in the title.

Hope this can be improved 🤞

Maybe something can be done our side also (like stripping apostrophe or changing the encoding or something).

rectifyer commented 5 years ago

It seems like the ’ character is the issue since that is different than a standard ' character. Hard to tell, but the first one is actually a back tick it looks like. The API returns the expected result when I used https://api.trakt.tv/search/movie\?query\=She%27s+All+That

tysonkerridge commented 5 years ago

I had a look into this issue also. It's definitely an issue with the character being used for an apostrophe. Unsure if the user is on iOS but I found that since iOS 11 the system keyboard has a "Smart Punctuation" setting that's on by default. This causes left/right single/double quotation mark characters to be used, such as ‘, ’, “, and ”, instead of standard apostrophe/quotation mark characters such as ' and ". It seems that the API is expecting apostrophe characters, and not a left (or right) single quotation mark, as its matching against those apostrophe characters in the content.

Based on that, I've found that you can toggle off some of the "Smart" features on individual UI components (eg. set smartQuotesType to UITextSmartQuotesType.no on a UISearchBar) which will then use apostrophes and the user won't have to worry. Alternatively could replace those characters before sending to the server etc. Not exactly ideal, I guess, but it "works" under the assumption that all API data is stored with apostrophe characters.

Maybe it's an overarching problem anyway, and I'd be interested to know if there's a way to make the problem go away at a server level, as it would affect all forms of text matching, especially with user text entry which becomes searchable, such as List names (eg. if I create a list called Tyson‘s List (left single quotation mark), it can't be found when searching for Tyson's List (apostrophe), or vice versa).

rectifyer commented 5 years ago

Great catch! I didn't realize iOS was doing that and switching the quotes to actually use back ticks. I think it makes sense to convert the back ticks to normal single and double quotes at the server level. I'll look into this more and see where I need to do this. To start, it would be search results, adding/editing lists, and adding/editing comments. Any other places you can think of?

tysonkerridge commented 5 years ago

Yeah I guess I'd noticed iOS was doing it but never knew why it was or when it started doing it, and never thought anything of it. I imagine they're probably using a bunch of other characters in similar ways but for different locales etc, eg. I've seen some instances of quotations using a „ (double low-9 quotation mark) for beginning quotes, eg. „This is a quote“. Did a quick search and found this: https://unicode-table.com/en/sets/quotation-marks/ which lists different types for different languages etc. And then yeah you have the ` (grave accent or back tick) which probably isn't entered unless specifically entered.

As for other places, I can't think of any other user input other than maybe the Checkin message parameter, but that probably doesn't require it to be edited.

rectifyer commented 5 years ago

I'll have a fix out for this later in the week. It will swap out the incorrect iOS backtick/qoute for a normal quote so the search query will return the correct results.