Atom 1.46.0 - Cannot load the checker for xx-YY

[ashthespy]

Description

Updating to atom v1.46.0 seems to have broken my spell-check.

The package `spell-check` cannot load the checker for `en-GB`. See the settings for ways of changing the languages used,  resolving missing dictionaries, or hiding this warning.

The plugin checked the following paths for dictionary files:
* C:\
* C:\Users\User\AppData\Local\atom\app-1.46.0\resources\app.asar.unpacked\node_modules\spellchecker\vendor\hunspell_dictionaries

On checking my system (Win 10), my dictionaries are where they normally are: %APPDATA%\Microsoft\Spelling which was picked up fine by the version of spell-check that was shipped in Atom 1.45..

Steps to Reproduce

1. Open a new Atom 1.46.0 instance
2. Get warning about dictionaries that can't be found..

Expected behavior:

Spell-check looks in system directories as it used to

Actual behavior:

Spell-check doesn't look in the right directory

Reproduces how often:

100%

Versions

> atom --version

 Atom    : 1.46.0
Electron: 4.2.7
Chrome  : 69.0.3497.128
Node    : 10.11.0

> apm --version
apm  2.4.3
npm  6.2.0
node 10.2.1 x64
atom 1.46.0
python 2.7.14
git 2.25.0.windows.1
visual studio 2015

> cmd /c ver
Microsoft Windows [Version 10.0.18363.778]

Additional Information

I'm guessing this is broken by #314?

[ashthespy]

As a follow up, setting my locale path manually to %APPDATA%\Microsoft\Spelling doesn't work with the follow stack trace..

Stack Trace

Failed to activate the spell-check package

At Cannot read property 'toLowerCase' of undefined

TypeError: Cannot read property 'toLowerCase' of undefined
    at Object.exports.getPath (/app.asar/node_modules/atom-pathspec/index.js:30:33)
    at LocaleChecker.deferredInit (/app.asar/node_modules/spell-check/lib/locale-checker.js:104:35)
    at LocaleChecker.check (/app.asar/node_modules/spell-check/lib/locale-checker.js:63:12)
    at SpellCheckerManager.check (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:995628)
    at Function.SpellCheckTask.startNextJob (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:14:917900)
    at SpellCheckTask.start (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:14:917152)
    at SpellCheckView.updateMisspellings (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:990885)
    at SpellCheckView.subscribeToBuffer (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:990192)
    at new SpellCheckView (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:988919)
    at ~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:240749
    at Workspace.observeTextEditors (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:428568)
    at Object.activate (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:240546)
    at Package.activateNow (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:3287350)
    at measure (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:3286762)
    at Package.measure (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:3284372)
    at activationPromise.activationPromise.Promise (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:3286622)
    at new Promise (<anonymous>)
    at Package.activate (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:3286565)
    at PackageManager.activatePackage (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:382700)
    at config.transactAsync (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:382316)
    at Config.transactAsync (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:331964)
    at PackageManager.activatePackages (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:382266)
    at PackageManager.activate (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:11:381816)
    at t.loadState.then (~/AppData/Local/atom/app-1.46.0/resources/app/static/<embedded>:1:743877)

[pschloss]

I'm getting the same error on Mac OS X 10.14.6 but for en-US

[ashthespy]

Well, it gets more interesting, I can silence the warning about the missing dictionary now, by unchecking the Use Locales - which leads to no errors on startup, but no linting either.

Explicitly pointing to my system dictionary directory also doesn't help either..

I assume there are the right dictionary files that spell-check requires, as this hasn't changed in a while..

...\Microsoft\Spelling\en-GB>ls

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       05/08/2019     11:20              2 default.acl
-a----       05/08/2019     11:20             24 default.dic
-a----       05/08/2019     11:20              2 default.exc

> Get-WinUserLanguageList
LanguageTag     : en-GB
Autonym         : English (United Kingdom)
EnglishName     : English
LocalizedName   : English (United Kingdom)
ScriptName      : Latin
InputMethodTips : {0809:A0000409}
Spellchecking   : True
Handwriting     : False

LanguageTag     : de-DE
Autonym         : Deutsch (Deutschland)
EnglishName     : German
LocalizedName   : German (Germany)
ScriptName      : Latin
InputMethodTips : {0407:00000407}
Spellchecking   : True
Handwriting     : False

@dmoonfire Any pointers?

[edwards-will]

Same error on Mac OS X 10.15.4 for en-US, Atom version 1.46.0

[pdxwolfy]

Same error here as well (same config as previous poster edwards-will)

[rayhendricks]

Have the same issue using macOS. Looks like the dependancies are out of date for this. Could this be causing issue?

[jpingle]

Same issue as well on my end (same config as edwards-will^). In addition the right click option to correct is still there and often crashes Atom, though this may be an unrelated issue.

[nupfel]

same here with atom nightly 1.48.0-nightly1 x64 on windows 10

[Starkovden]

same problem I see today atom 1.46.0 on windows 10

[ashthespy]

@lkashef Any pointers on how to roll back to the previous version until this is investigated?

[nupfel]

@ashthespy I have tried to rollback just spell-check using apm install spell-check@0.75.0 but it didn't work for me under windows 10 because it tried to compile something lacking the .NET framework 2 SDK, but if you just install atom 1.45.0 and prevent auto updates it should just work.

[dmoonfire]

It looks like there are a number of issues:

@ashthespy: The Locale checker (as did the previous one) does require the dictionary files to be located in a folder with the locale name being the base name for the files. For example, on Linux, I have:

$ ls /usr/share/hunspell/
de_BE.aff  de_DE.dic  en_US.aff  fr.dic     fr_CA.aff  fr_CH.dic  fr_LU.aff  fr_MC.dic  ru_RU.aff
de_BE.dic  de_LU.aff  en_US.dic  fr_BE.aff  fr_CA.dic  fr_FR.aff  fr_LU.dic  pl_PL.aff  ru_RU.dic
de_DE.aff  de_LU.dic  fr.aff     fr_BE.dic  fr_CH.aff  fr_FR.dic  fr_MC.aff  pl_PL.dic

The base library also doesn't know how to read Microsoft's dictionary which is why pointing it to there didn't work (plus it was looking for %APPDATA%\Microsoft\Spelling\en-GB.dic and it can't handle the % variables either).

Also, the core node-spelling library only provides en-US locale (that second path listed).

The reason we split the system and locale checkers apart was because we found some Windows installations didn't allow you to change languages. You seem to be one of those cases since you have "System" checked (use default Windows) or it's a case I haven't seen yet.

Okay, to fix it: If you have Libreoffice or OpenOffice, you can set your Locale folder to the path of those dictionaries. If not, then create a folder somewhere and copy the en-GB.* files from an online source such as https://github.com/LibreOffice/dictionaries/tree/master/en and stick those files into that path. It will then look up C:\some\path\to\dictionaries\en-US.dic and hopefully do the right thing and work.

@pschloss, @edwards-will, @pdxwolfy, @jpingle: For Mac, the Locale checker shouldn't be required. Could you uncheck that, make sure the system checker is checked, and tell me if you still get spelling? The system checker is supposed to use the Mac's built-in spell-checker with whatever languages you have defined.

General: I need to investigate why the error is happening still. That stack trace is helpful, I just need to trace through code since naturally it doesn't happen for me. :(

[ashthespy]

The base library also doesn't know how to read Microsoft's dictionary which is why pointing it to there didn't work (plus it was looking for %APPDATA%\Microsoft\Spelling\en-GB.dic and it can't handle the % variables either). The reason we split the system and locale checkers apart was because we found some Windows installations didn't allow you to change languages. You seem to be one of those cases since you have "System" checked (use default Windows) or it's a case I haven't seen yet.

I figured from the trace that it couldn't read the ENV variable, but even with the full path(C:\...\Roaming\Microsoft\Spelling\en-GB), it isn't reading my system dictionaries. If I understand you correctly, Locale Paths is only used to read hunspel dictionaries, and not the system dictionaries?

In the case of Use System - where does it try and pull the system dictionary from? I had a quick look at atom/node-spellchecker and it does seem to indicate it can use native Windows spell checking via the Windows Spell Check API? Casue IIRC, I didn't require hunspel dictionaries prior to this, and it used my system installed dictionaries automatically..

[dmoonfire]

Use System always uses the Windows Spell Check API. That was related to the recent split. So in your case, you should be able to have Use System checked and Use Locales unchecked and have it work.

I have found some environments (my work machine actually), where I couldn't choose anything other than US English for my system language (thanks to my IT locking down my machine). So, splitting the Use Locale was broken out more formally to handle those situations too, where you want a dictionary that you don't have installed in Windows (or can't install the spelling module).

Previously, when the code had the two integrated, it would ignore the locales code on Windows 10 and just use the system.

Edit: On Mac, Use System always uses the Apple spelling API. On Linux, Use System does nothing.

[pdxwolfy]

On a Mac, disabling "Use Locales" seems to have fixed the issue - no more error message and spell checking appears to work.

[ashthespy]

Use System always uses the Windows Spell Check API. That was related to the recent split. So in your case, you should be able to have Use System checked and Use Locales unchecked and have it work.

This (https://github.com/atom/spell-check/issues/328#issuecomment-624604505) leads to no warning about missing locale dictionary, but also no spell checks.. From https://github.com/atom/spell-check/pull/314#issuecomment-571222100 and my testes previously it would seem that the Use System doesn't seem to be doing its job, and just silently failing? Any pointers on how to debug?

Edit: Another data point - on Atom 1.45.0 with spell-check 0.75.0 things are working as expected with en-GB being selected from what I can see:

[dmoonfire]

Give me a couple hours? I can write a test version of spell-check that has additional debugging statements that might help me track it down. I'm not sure what of the two packages is having the problem.

[dmoonfire]

@ashthespy Could you use https://github.com/dmoonfire/spell-check/tree/locale-issues in your local version? It has debug installed with some debugging. To enable the logs, go into the console and type:

localStorage.debug = 'spell-check:*'

Then reload your window (Control-Shift-R). Once you do that, you should see a bunch of colorful messages that will hopefully give me some ideas of what is going on.

Thank you.

[ashthespy]

@dmoonfire Thanks for that, but apm doesn't want to build spellchecker for me.

Building spellchecker

``` C:\Users\Ash\.atom\dev\packages\spell-check [locale-issues ≡]> apm install Installing modules failed > spellchecker@3.7.0 install C:\Users\Ash\.atom\dev\packages\spell-check\node_modules\spellchecker > node-gyp rebuild C:\Users\Ash\.atom\dev\packages\spell-check\node_modules\spellchecker>if not defined npm_config_node_gyp (node "C:\Users\Ash\AppData\Local\atom\app-1.46 .0\resources\app\apm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "C:\Us ers\Ash\AppData\Local\atom\app-1.46.0\resources\app\apm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild ) Warning: Missing input files: C:\Users\Ash\.atom\dev\packages\spell-check\node_modules\spellchecker\build\..\vendor\hunspell\src\hunspell\utf_info.hxx Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch. C:\Users\Ash\.atom\dev\packages\spell-check\node_modules\spellchecker\build\hunspell.vcxproj(20,3): error MSB4019: The imported project "C:\Microsoft.Cp p.Default.props" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk. gyp ERR! build error gyp ERR! stack Error: `C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe` failed with exit code: 1 gyp ERR! stack at ChildProcess.onExit (C:\Users\Ash\AppData\Local\atom\app-1.46.0\resources\app\apm\node_modules\npm\node_modules\node-gyp\lib\build .js:262:23) gyp ERR! stack at ChildProcess.emit (events.js:182:13) gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12) gyp ERR! System Windows_NT 10.0.18363 gyp ERR! command "C:\\Users\\Ash\\AppData\\Local\\atom\\app-1.46.0\\resources\\app\\apm\\bin\\node.exe" "C:\\Users\\Ash\\AppData\\Local\\atom\\app-1.46. 0\\resources\\app\\apm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild" gyp ERR! cwd C:\Users\Ash\.atom\dev\packages\spell-check\node_modules\spellchecker gyp ERR! node -v v10.2.1 gyp ERR! node-gyp -v v3.7.0 gyp ERR! not ok npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! spellchecker@3.7.0 install: `node-gyp rebuild` npm ERR! Exit status 1 npm ERR! Failed at the spellchecker@3.7.0 install script. ``` Not sure how to get `apm` to use my Visual Studio 2017 build tools, instead of `msbuild`. `npm` has no such issues, and built the module fine, but atom doesn't load it as it's built for my systems node version. `apm config set msvs_version 2017` doesn't seem to do anything for me.. EDIT: Got it - had to set [`$env:GYP_MSVS_VERSION=2017`](https://github.com/atom/apm/blob/adb72dc9ca7563f060f00d0f652eb4e641b852d8/src/apm.coffee#L86) I can't seem to make atom use the `spell-checker` in dev mode, as it now complains about missing `debug` module. If I run `apm install` as usual, it complains that `spellcheck` is the wrong native version. I will try again in some time..

Okay, managed to get it working by linking the pre-compiled spellcheck from the Atom installation. This is what I get now..

spell-check:system-checker enabled +0ms true working correctly
system-checker.coffee:44 Uncaught (in promise) TypeError: this.log is not a function
    at system-checker.coffee:44

Fixed that error with a fat arrow(.then (incorrect) =>, but it looks like it's just failing silently:

i.e incorrect is always empty..

[dmoonfire]

(Sorry about the fat arrow. My next task in spell-check is to decaffinate it to avoid those mistakes.)

Okay, since check() isn't returning bad values, that points that the underlying library isn't returning good results. Give me a bit (can't code a fix during work hours) and I'll see if I can give you a different tests case that changes the inputs around. One of the things we added was the ability to say "always use Hunspell", "always use system API", or "make your best choice". There is a chance that is the problem.

[dmoonfire]

@ashthespy I updated with a hacky code to see if I can get a better answer. Please try it again? Same thing?Look at the debug.

[ashthespy]

@dmoonfire Thanks for all your efforts!

[dmoonfire]

@ashthespy: I was afraid of that response. It looks like the two basic ways of initializing the library both don't return results for you. This problem with probably have to be delved into the next library (https://github.com/atom/node-spellchecker).

Since you are having trouble building the projects, I have one more test that might find the problem. I added some if checks since I don't know if if we have a process/global pollution. They should all show [ { start: 10, end: 17 } ] as the error. Maybe try some various true/false and see if you get different results or at least ones that consistently work?

// Setup:
//   Create a test folder and copy this file into it.
//   yarn add spellchecker (or npm install spellchecker)
//   node test.js

const SpellChecker = require("spellchecker");
const test = "with some speling";
const testStatic = true;
const testDefault = true;
const testDict = true;
const testPath = true;
const testAuto = true;
const testSystem = true;

if (testStatic) {
    console.log("dict   ", SpellChecker.getAvailableDictionaries());
    console.log("path   ", SpellChecker.getDictionaryPath());
    console.log("static ", SpellChecker.checkSpelling(test));
}

if (testDefault) {
    const checker = new SpellChecker.Spellchecker();
    console.log("default", checker.checkSpelling(test));
}

if (testPath) {
    const checker = new SpellChecker.Spellchecker();
    console.log("en-GB? ", checker.setDictionary("en-GB", ""));
    console.log("en-GB  ", checker.checkSpelling(test));

    const checker2 = new SpellChecker.Spellchecker();
    console.log("en_GB? ", checker2.setDictionary("en_GB", ""));
    console.log("en_GB  ", checker.checkSpelling(test));
}

if (testAuto) {
    const checker = new SpellChecker.Spellchecker();
    checker.setSpellcheckerType(SpellChecker.USE_SYSTEM_DEFAULTS);
    console.log("auto   ", checker.checkSpelling(test));
}

if (testSystem) {
    const checker = new SpellChecker.Spellchecker();
    checker.setSpellcheckerType(SpellChecker.ALWAYS_USE_SYSTEM);
    console.log("system ", checker.checkSpelling(test));
}

What I'm looking for is what settings I need to make it work for your environment. I'm hoping there is an option, otherwise, I may need to get you working with a build environment so you can compile a printf-encumbered version of node-spellchecker (which no one wants).

Thank you for helping with this.

[ashthespy]

@dmoonfire It would seem that spellchecker isn't able to pick up my system dictionaries..

testStatic
dict    [
  'de-AT', 'de-CH', 'de-DE',
  'de-LI', 'de-LU', 'en-029',
  'en-AU', 'en-BZ', 'en-CA',
  'en-GB', 'en-HK', 'en-ID',
  'en-IE', 'en-IN', 'en-JM',
  'en-LR', 'en-MY', 'en-NZ',
  'en-PH', 'en-SG', 'en-TT',
  'en-US', 'en-ZA', 'en-ZW',
  'nl-BE', 'nl-NL'
]
path    C:\Users\Ash\.atom\dev\packages\spellchecker\node_modules\spellchecker\vendor\hunspell_dictionaries
static  [ { start: 10, end: 17 } ]

testDefault
default []

testPath
en-GB?  true
en-GB   [ { start: 10, end: 17 } ]
en_GB?  true
en_GB   [ { start: 10, end: 17 } ]

testAuto
auto    []

testSystem
system  []

Could it be that this was always the case, and the issue went unnoticed as en-GB is included in the pre-packaged dictionaries from spellcheker ?

[dmoonfire]

No, that's awesome. Now, let me figure out how to juggle things around to make it actually work.

Basically, the testPaths tests the auto feature (use system verses use hunspell), provides a language, but does not provide a path. In this situation, it uses the Windows API but requires us to specify a language code which is different than Mac which requires using the built-in checker but cannot specify a language (or it crashes).

The dict line are your system dictionaries. spellchecker only packages en-US and the getDictionaries call queries the spelling API to get a list.

[dmoonfire]

@ashthespy: If you would be so kind, I have updated my repository with a new version of the code. I'm coding a bit blind (the tests are clean on my machine), but this version does the following:

• If we can't find the dictionary through the paths, try the testPaths version in the sample script.
• If we inferred the locale and have a system checker, we don't show a warning message.
• Expanded documentation in the read me.

[ashthespy]

@dmoonfire I spend a few minutes looking into this and here is a brain dump. First off, with https://github.com/dmoonfire/spell-check/commit/a71b74fd94f9211e9776dab2334eb217488185b7 I realised that I wasn't getting any of the additional debug statements that you added. After fumbling around, found https://github.com/atom/spell-check/blob/8e7f796580f6461e8105901b6ab38d8e30c39e83/lib/spell-check-manager.coffee#L332-L333 Which meant that the no LocaleChecker was being created in my case(Only Use System checked, nothing else). Flipping the and to an or solves that, and now the LocaleChecker is up and running.

But still no dice with the checking. From the logs, the right locale is being selected (from navigator.language in my case) and the appropriate dictionary is being selected: https://github.com/atom/spell-check/blob/8e7f796580f6461e8105901b6ab38d8e30c39e83/lib/locale-checker.coffee#L91-L94

But, looking at SystemChecker again we see: https://github.com/atom/spell-check/blob/8e7f796580f6461e8105901b6ab38d8e30c39e83/lib/system-checker.coffee#L5-L8

And sure enough, with

diff --git a/lib/system-checker.coffee b/lib/system-checker.coffee
index fd2a8e9..6a62d7b 100644
--- a/lib/system-checker.coffee
+++ b/lib/system-checker.coffee
@@ -17,7 +17,8 @@ class SystemChecker
   constructor: ->
     @log = debug('spell-check:system-checker')
     @log 'enabled', @isEnabled(), @getStatus()
-
+    @log 'Setting en-GB manualy:' , instance.setDictionary("en-GB", "")

   deactivate: ->
     return

Basically, the testPaths tests the auto feature (use system verses use hunspell), provides a language, but does not provide a path. In this situation, it uses the Windows API but requires us to specify a language code which is different than Mac which requires using the built-in checker but cannot specify a language (or it crashes).

tl;dr

• Spellchecker on Windows doesn't initialise a "default" dictionary.
• LocaleChecker is never initialised, but is needed for Windows.
• SystemChecker initialises a new Spellchecker and thus setDictionary calls from LocaleChecker are moot, but required for Windows.

[dmoonfire]

@ashthespy: Please try turning "Use Locales" on with my code. What I added was that setDictionary("en-GB", "") call in there, there are other places that need that flipped to actually use the locale dictionary (your changing it to or briefly bypassed that but only for initialization).

[ashthespy]

@dmoonfire Ah - didn't go through the git history, didn't bother flipping the settings for the dev version. Your indeed correct - setting Use Locales Just Works™ with your tree!

[dmoonfire]

@pschloss, @pdxwolfy, @Starkovden, @nupfel: If you are able, would you verify the fixes I have on my branch work?

https://github.com/dmoonfire/spell-check/commit/a71b74fd94f9211e9776dab2334eb217488185b7

On Mac and Windows, you should have it start up and do spelling if you have "Use System" and "Use Locales" checked and nothing in the "Locales" field.

On Windows, and not using a default language, putting just the IEFT tag in "Locales" should work too.

Thank you.

[pdxwolfy]

I'm not sure how to install the branch, but using System and unchecking Locale worked for me on my Mac.

[dmoonfire]

@pdxwolfy: I added a feature that if you had "Use System" and "Use Locales", it will stop complaining if it can't use locales.

[ashthespy]

On Mac and Windows, you should have it start up and do spelling if you have "Use System" and "Use Locales" checked and nothing in the "Locales" field.

Hmm not in my case - I have to uncheck "Use System" for it to work..

On Windows, and not using a default language, putting just the IEFT tag in "Locales" should work too.

This works :-) (but again Use System has to be unchecked)

I guess this is because with Use System checked, a new SpellCheker instance is created that has no dictionary set, that takes precedence over the instance created in locale-checker with the right dictionary set for Windows?

[jaryth000]

Just to echo was @ashthespy mentioned. Started up Atom today and realized my spellchecker was not working at all. I was not getting any errors, and disabling Use Locales did not fix the issue, but leaving Use Locales enabled, with a blank Locales box, and then disabling Use System did fix the issue for me. If it means anything, I had this issue just after installing Windows Update's KB4556799 + KB4552931.

[davesag]

Still seeing this issue in 1.47.0 on macOS. Deactivating 'use locales' in the spell-check prefs fixes it.

[tobypeschel]

Same issue on 1.51.0 on Linux (Manjaro) with en-GB: The package spell-check cannot load the checker for en-GB.

[dmoonfire]

@tobypeschel, this would work better with a new ticket instead of this one. But, do you have Manjaro's en-GB dictionaries installed and have you tried putting the directory that contains the en_GB.dic (or something like that) in the locales path?