[EnhancementReq] Ignore diacritics in Quick Filter

[Hakun1n]

Hi,

Is there any way to make a switch for QuickSearch to ignore diacritics? I could even live with a manually configurable replacement table (like ... consider "č" as "c", etc...).

This is one of the most-annoying thing in Thunderbird for me. My native language has a lot of diacritics and since the SMS era it is quite common that people are not using diacritics in informal emails/messages (it used to be cheaper as we had char-limits :p). Especially for people in IT and/or for those for which English (or other lang) is the daily driver. But the other half is using diacritics fully.

Quick-filter is absolute blast when i KNOW that i recently got certain email (eg. i saw it on my cell, but want to reply on PC). So i just open Thunder/Better-bird, click to QuickFilter, type the word and ... nothing ... (maybe i did a typo? second attempt!) ... nada .. then i realize that the freaking word has a "ž" in it, so i either have to switch keyboard and quick-search for third time OR i have to use the global-search (which is an overkill for such case BUT it actually ignores diacritics). Worst-case is if the word has multiple letters with diacritics but the sender forgets one ... then it is a guess-game till i give up and use the global search.

There is a related ancient bug-report in bugzilla - 506064 closed as duplicate to even more ancient 284856 (19 years)

Note: I tried the Expression Search - NG which allows using "g " to invoke global search but it is clunky, it has annoying history overlay and the fact a prefix is required voids the "Quick" meaning.

Thank you in advance for at least reading this :)

[Betterbird]

How about using regular folder search together with regular expression matching? Matches "xx.yy" will match "xxayy" and "xxäyy".

Otherwise we'd have to implement some very expensive diacritics matching. That said, if you search in this page in Firefox, both ä and a will match the a's on the page. We'd have so see how this is done and whether it can be adapted to BB.

Would you like to fund the enhancement?

[Betterbird]

Looks like the Mozilla "find in page" happens here: https://searchfox.org/mozilla-central/rev/863aef60f9fa1536884252ad90cad1d8ee9d8a41/toolkit/components/find/nsFind.cpp#585 Look for mMatchDiacritics in that code. That can't be used for quick filter.

ICU string search could be used for string search ignoring diacritic: https://unicode-org.github.io/icu/userguide/collation/string-search.html

This would need to be plugged in here into nsMsgSearchTerm::MatchString(): https://searchfox.org/comm-central/rev/c3c7f562cb0e1d0b409c81e9f3cbf46f10f73c02/mailnews/search/src/nsMsgSearchTerm.cpp#976

Looks the TB folks track it in https://bugzilla.mozilla.org/show_bug.cgi?id=1839085.

No idea why they duped https://bugzilla.mozilla.org/show_bug.cgi?id=506064 to https://bugzilla.mozilla.org/show_bug.cgi?id=284856 since the latter appears to be IMAP search on the server, at least according to the last comment.

[Hakun1n]

How about using regular folder search together with regular expression matching? Matches "xx.yy" will match "xxayy" and "xxäyy".

That would be OK-ish to "avoid" one letter but quite inconvenient with multiple letters. Especially when the remaining piece is a base for several other words.

Otherwise we'd have to implement some very expensive diacritics matching. That said, if you search in this page in Firefox, both ä and a will match the a's on the page. We'd have so see how this is done and whether it can be adapted to BB.

Yes, Firefox has a checkbox "Match Diacritics" (un-checked by default) and that is working like a charm. But i see in the other reply that it can not be used here

Would you like to fund the enhancement?

Depends on "how much" ? I could make some "Donation" but if we're talking some significant amount then i might just pass. Hard to say as I can't evaluate the real effort needed to get this implemented. I can drop you an email if you wish. Also if there are more people willing to support this that would be nice.

BTW I discovered Betterbird recently and by complete coincidence. I was looking for something TB-related and someone mentioned your fork on Reddit. You really need more PR :) .

Cheers!

[Betterbird]

I can drop you an email if you wish.

Yes, please.

[Betterbird]

https://github.com/Betterbird/thunderbird-patches/commit/f6b606f43a2de2e62044c7e20b9016d85029cdfb https://github.com/Betterbird/thunderbird-patches/commit/0d1d3ce6d7f4aba273fe38047addc4a6d4de163d

[Hakun1n]

f6b606f 0d1d3ce

That was super fast. I've checked the version 115.10.0-bb27 but sadly I am not able to make it work. Quick filter of zadost does not find žádost nor žadost. Do i need to "enable" the feature somewhere?

Checking this line it already contains ž so that should not be an issue. On the other side there are some characters (from my lang) which are missing -> čČřŘďĎťŤňŇÝůŮ

Oh and the email has been sent.

[Betterbird]

Since it was requested after the 115.10.0 release, it's not in that release. Please try: https://www.betterbird.eu/downloads/WindowsInstaller/betterbird-115.10.0-bb27-latest-build1.en-US.win64.installer.exe

The second changeset is relevant since there we switched the processing to ICU to cover all possible accents, see: https://en.wikipedia.org/wiki/Diacritic#Languages_with_letters_containing_diacritics

[Hakun1n]

Oh, silly me. I missed the disclaimer on the Release Notes page. I've now updated to the latest build and it works like a charm. You made my day, thanks !