Spellcheck libraries not detected (Mac) (3/4 issues identified)

[andrei-akopian]

Device: M1 Mac Air MacOS version: Ventura 13.4 Python version: python 3.11.4

Manuskript isn't detecting any of the libraries it asked me to download for the spellcheck. And guide on the wiki doesn't seem to be helpful for my case, as it only covers installing enchant dictionaries.

I am suspecting this is something about python versions. pandoc wasn't detecting either so I put in the path manually.

Edit:

Pandoc should probably be a different issue, since all the issues here were for different reasons.

As it says in the title I have identified the issues with 3/4 libraries to a pretty high degree. And I did get Spellcheck to work using pyenchant, but I don't think this issue is truly solved, and should be kept open.

The reason the libraries are displayed as "need to be installed" is because the imports in the try statements at the beginning of spellchecker.py are giving an error, that is also the file/script I was looking at when trying to figure out the issues (especially the one with symspellpy)

All the conclusions/summery are in my final comment.

[andrei-akopian]

I have worked on solving my problem for a bit, the enchant import fails because ImportError: The 'enchant' C library was not found and maybe needs to be installed. this is because my computer for some reason can't see the C enchant libraries in /opt/homebrew/lib/enchant-2 (homebrew) and /opt/local/lib/enchant-2 (macports)

[andrei-akopian]

pyspellchecker can be imported just fine in the terminal (and simple test scripts), but it doesn't seem to work correctly for me (print(spell.correction('helo')) prints "helo") and manuscript still can't see it.

[andrei-akopian]

After a bit of tinkering I figured out why symspellpy doesn't work:

The library responsible for checking if the symspellpy library has a high version (distutils.version library) is actually throwing an error:

DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if distutils.version.LooseVersion(symspellpy.__version__) < SYMSPELLPY_MIN_VERSION:

Snippet from spellchecker.py

SYMSPELLPY_MIN_VERSION = "6.3.8"
try:
    import symspellpy
    import distutils.version

    if distutils.version.LooseVersion(symspellpy.__version__) < SYMSPELLPY_MIN_VERSION:
        symspellpy = None

except ImportError:
    symspellpy = None

For some reason the error is suggesting to use packaging.version library, HOWEVER packaging.version doesn't even work. And after some more testing, I can't understand the reason distutils.version is even necessary, the code:

SYMSPELLPY_MIN_VERSION = "6.3.8"
try:
    import symspellpy

    if symspellpy.__version__ < SYMSPELLPY_MIN_VERSION:
        symspellpy = None
    print('"5.9.9" < SYMSPELLPY_MIN_VERSION results',"5.9.9" < SYMSPELLPY_MIN_VERSION)
    print('"6.7.7" < SYMSPELLPY_MIN_VERSION results',"6.7.7" < SYMSPELLPY_MIN_VERSION)

except ImportError:
    symspellpy = None

print(symspellpy)

Outputs the desired result

"5.9.9" < SYMSPELLPY_MIN_VERSION results True
"6.7.7" < SYMSPELLPY_MIN_VERSION results False
<module 'symspellpy' from '/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/symspellpy/__init__.py'>

[andrei-akopian]

I have figured out the issue with enchant apparently it is a common issue on M1 Macs with no easy solution (at least none that I understood.) The issue is described in https://github.com/pyenchant/pyenchant/issues/265

[andrei-akopian]

I got LanguageTool to work. Apparently installing language-tool-python wasn't enough. As far as I understand the language-tool-python library is only a wrapper and I didn't have the actual LanguageTool installed. After I ran a test script:

import language_tool_python

tool = language_tool_python.LanguageTool('en-US')

text = 'Hello, how are yuo doing?'

matches = tool.check(text)

assert len(matches) == 1
assert matches[0].ruleId == 'MORFOLOGIK_RULE_EN_US'

print("LanguageTool is working properly!")

The "real" 200MB LanguageTool got installed automatically and it worked fine from there:

[andrei-akopian]

I have figured out the issue with enchant apparently it is a common issue on M1 Macs with no easy solution (at least none that I understood.) The issue is described in pyenchant/pyenchant#265

Looking at it again I realized that I am a bad tester:

As you can see the spelling correction is working using pyenchant (for some odd reason)

I randomly pasted a bunch of stuff from pyenchant/pyenchant#265 into the command line (Installed the Intel version of brew and installed the enchant C library using it), and I ended up with even more broken pyenchant (I ran multiple python scripts for testing and and they all gave import errors because of some architecture issues).

Somehow while all my pyenchant_test.py scripts were completely broken, the Spellcheck started to work (using pyenchant) without me noticing, after that I did the tinkering with LanguageTool described in the comment above.

[andrei-akopian]

Conclusion:

---

PyEnchant (issue) controversy started to work after I "completely broke it" as described here: Comment above

Symspellpy actually works just fine but is failing the try statement because the library used to check the version is deprecated (that doesn't even seem to be needed.) Described in this comment: broken distutils.version

Pyspellchecker: not sure what's wrong here.

LanguageTool: language-tool-python library is only a wrapper and the actual LanguageTool must be installed (it auto-installs by when running a test script) Full description here

---

Because pyenchant decided to just start working for some unknown reason, I couldn't figure out whether my fix for LanguageTool made it actually work, and what was wrong with pyspellchecker. I suggest we keep this issue open at least until Symspellpy issue is fixed.

[TheJackiMonster]

Very interesting. Maybe we can improve the next release with all this information.

I think language-tool-python should also install LanguageTool automatically when using it in Manuskript. So the script should not be necessary. But I think I need to verify that on a fresh install.

Thanks for the investigation and the detailed write up already!