ISRI Arabic Stemmer broken on Python >= 3.6

[b2m]

The ISRI ARabic Stemmer (src.whoosh.lang.isry.py) does not work on Python >= 3.6.

Exception: re.error: bad escape \u at position 0. Reason: changed behavior of re.sub.

Changed in version 3.6: Unknown escapes in pattern consisting of '\' and an ASCII letter now are errors.

(quoted from https://docs.python.org/3/library/re.html)

Code snippet to reproduce:

from whoosh.analysis import LanguageAnalyzer

analyzer = LanguageAnalyzer(lang='ar')
[(token.text, token.stopped) for token in analyzer("This is a test")]

Codesamples with bad escape sequences:

• https://github.com/whoosh-community/whoosh/blob/26153e270f3534ea19e2fc97a54a3477c937a500/src/whoosh/lang/isri.py#L87
• https://github.com/whoosh-community/whoosh/blob/26153e270f3534ea19e2fc97a54a3477c937a500/src/whoosh/lang/isri.py#L88
• https://github.com/whoosh-community/whoosh/blob/26153e270f3534ea19e2fc97a54a3477c937a500/src/whoosh/lang/isri.py#L89
• https://github.com/whoosh-community/whoosh/blob/26153e270f3534ea19e2fc97a54a3477c937a500/src/whoosh/lang/isri.py#L190
• https://github.com/whoosh-community/whoosh/blob/26153e270f3534ea19e2fc97a54a3477c937a500/src/whoosh/lang/isri.py#L194

[stevennic]

Thanks Benjamin, good catch. I have submitted a PR to fix this.

[nijel]

Should be fixed by https://github.com/whoosh-community/whoosh/pull/557