Reorder search results

[ageorgou]

From the meeting on 1 May:

Sort in search table to be changed so it displays hyphenated entried last. Also order of s to be: s, s., s^ (similar for t-variants).

Essentially:

• We need to make sure that non-English alphabet characters appear close to their base versions, according to the order above (this affects s, t, and some vowels e.g. a-bar, a_2?)
• We should assign less priority to fragmented words, so they appear further down in the list of results. This can be done by sorting "-" after other characters.

The last point could be clarified a bit. For example, does AB-C come after ADE? In other words, does any "-" make the word appear at the end, or is it just a character sorted after z?

[ageorgou]

Some notes from looking into how we can do this without coding from scratch:

• The localeCompare method of strings is useful, but by default it sorts š before ş, which is not the order we want. Its behaviour may also depend on the user's browser (and location? but I think not).
• We can specify a particular locale to use, but that is not supported on all browsers (of note, Chrome on mobile), and only offers a predefined set of choices (which I guess does not include Sumerian dialects...)
• We can maybe specify a custom locale (this answer seems to suggest so, but from a quick look I don't see how). It may be useful to look at Collator more generally, or how it's done in e.g. this project.

Failing this, we can just build the string comparison from scratch based on our desired ordering.

[ageorgou]

We need to check which non-ASCII characters we want to support; the Oracc docs include more characters than the one mentioned above.

[ageorgou]

On the back-end side, the ICU plugin for ElasticSearch may be of interest.

[ageorgou]

Putting this here because it's surprisingly hard to search for: The Unicode default collation chart (DUCET - Default Unicode Collation Element Table): http://unicode.org/charts/collation/. This is the sorting currently chosen in the backend.

[tim-band]

So, what do we actually want now? We had plain ASCII searching, which was bad, now we're back to English collation. Perhaps we want to adjust it somewhat? Or maybe it's fine?