kiwix-serve raises 500 error on `Or` search term

[benoit74]

See e.g. https://library.kiwix.org/catalog/v2/entries?start=0&count=4&lang=fra&category=other&q=Or

[veloman-yunkan]

The problem here is that the search text is interpreted as a Xapian query where or is an operator, and therefore the query is syntactically invalid. Note that if you search for "composition or rose" (without quotes) it will be interpreted as a query for either composition or rose rather than a query for all of composition, or and rose.

There are different ways to address this issue:

1. Preprocess the search text in the front end so that words coinciding with Xapian query operators are somehow escaped (e.g. quoted or preceded with a +). The backend and the semantics of the q parameter of the /catalog/v2/entries API endpoint remain unchanged (thus it will be possible to pass advanced Xapian queries via the HTTP API).
2. Redefine the syntax and semantics of the q parameter of the /catalog/v2/entries API endpoint and parse it respectively in the backend. The content of the searchbox is passed to the endpoint as is (like in the current implementation).
3. Document the current implementation and fix it so that a proper error is displayed to the user. The users will have to escape their queries on their own.

[veloman-yunkan]

@kelson42 @rgaudin ping

[kelson42]

This issue remember me of https://github.com/kiwix/kiwix-tools/issues/440, I need to make a reassessment of all of this.

[kelson42]

@veloman-yunkan Solution (1) - so a kind of xapian_escaping() should be implemented, but this should be done IMHO:

• At the reader side (so not deep in the search backend)
• Potentialy in all readers
• Via a new public method (in libzim?)

For Kiwix serve, not sure exactly how it should be done... but I guess this problem is potentialy everywhere