search

languagetool-org / languagetool

Style and Grammar Checker for 25+ Languages
https://languagetool.org
GNU Lesser General Public License v2.1
12.32k stars 1.39k forks source link

[en] Maven broken build, LanguageTool-20240216-snapshot.zip testrules error #10327

Closed MikeUnwalla closed 6 months ago

MikeUnwalla commented 8 months ago

From Maven:

[INFO]
[ERROR] Errors:
[ERROR]   DynamicMorfologikLanguageTest.test:35 » Runtime java.lang.RuntimeException: Could not check sentence (language: Testlang): <sentcontent>test</sentcontent>
[INFO]
[ERROR] Tests run: 115, Failures: 0, Errors: 1, Skipped: 8
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for LanguageTool 6.4-SNAPSHOT:
[INFO]
[INFO] LanguageTool ....................................... SUCCESS [  0.114 s]
[INFO] LanguageTool Style and Grammar Checker Core ........ SUCCESS [ 12.641 s]
[INFO] English module for LanguageTool .................... FAILURE [ 43.148 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  56.166 s
[INFO] Finished at: 2024-02-17T09:43:38Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0:test (default-test) on project language-en:

Testrules (first part) for standalone LanguageTool 2024-02-16:

<snip>\LanguageTool-6.4-SNAPSHOT>testrules en
Running XML pattern tests...
LanguageTool version 6.4-SNAPSHOT (2024-02-16 18:55:08 +0100, a71f133)
Known languages: [Arabic, Asturian, Belarusian, Breton, Catalan, Catalan (Valencian), Catalan (Balearic), Danish, German, German (Germany), German (Austria), German (Swiss), Simple German, Greek, English, English (US), English (GB), English (Australian), English (Canadian), English (New Zealand), English (South African), Esperanto, Spanish, Spanish (voseo), Persian, French, French (Canada), French (Switzerland), French (Belgium), Irish, Galician, Italian, Japanese, Khmer, Dutch, Dutch (Belgium), Polish, Portuguese, Portuguese (Portugal), Portuguese (Brazil), Portuguese (Angola preAO), Portuguese (Moçambique preAO), Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Tagalog, Ukrainian, Chinese, Crimean Tatar, Testlanguage]
Running XML validation for en/grammar.xml...
Running XML validation for en/style.xml...
Running XML validation for /org/languagetool/rules/en/remote-rule-filters.xml...
Running pattern rule tests for English (org.languagetool.language.English)...
Exception in thread "main" java.lang.RuntimeException: Could not activate rules
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:94)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:84)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:67)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:51)
        at org.languagetool.rules.patterns.PatternRuleTest.createToolForTesting(PatternRuleTest.java:210)
        at org.languagetool.rules.patterns.PatternRuleTest.runTestForLanguage(PatternRuleTest.java:192)
        at org.languagetool.rules.patterns.PatternRuleTest.runGrammarRulesFromXmlTestIgnoringLanguages(PatternRuleTest.java:184)
        at org.languagetool.rules.patterns.PatternRuleTest.main(PatternRuleTest.java:927)
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml'
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
        at org.languagetool.Language.getPatternRules(Language.java:705)
        at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
        ... 9 more
Caused by: java.lang.RuntimeException: Could not create filter class using constructor public org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter() throws java.io.IOException
evan-defran-lt commented 8 months ago

I am unable to reproduce the error that you received:

 Results:
[INFO] 
[WARNING] Tests run: 115, Failures: 0, Errors: 0, Skipped: 8
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for LanguageTool 6.4-SNAPSHOT:
[INFO] 
[INFO] LanguageTool ....................................... SUCCESS [  2.614 s]
[INFO] LanguageTool Style and Grammar Checker Core ........ SUCCESS [02:18 min]
[INFO] English module for LanguageTool .................... SUCCESS [05:33 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  07:59 min
[INFO] Finished at: 2024-02-17T11:34:41+01:00
[INFO] ------------------------------------------------------------------------

Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml' ^ This typically indicates a syntax error in the XML code. Without knowing much more about what you are seeing, I can only suggest that the surefire-report for EnglishPatternRuleTest might indicate where the error occurs (if there is indeed a syntax error).

MikeUnwalla commented 8 months ago

I tried Maven again. This is (I think) the applicable Maven message:

[INFO] Running org.languagetool.DynamicMorfologikLanguageTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.051 s <<< FAILURE! - in org.languagetool.DynamicMorfologikLanguageTest
[ERROR] org.languagetool.DynamicMorfologikLanguageTest.test  Time elapsed: 0.05 s  <<< ERROR!

I attach org.languagetool.DynamicMorfologikLanguageTest.txt: org.languagetool.DynamicMorfologikLanguageTest.txt

evan-defran-lt commented 8 months ago

@jaumeortola, do you know anything about this?

jaumeortola commented 8 months ago

I cannot reproduce the problem on Linux. According to the attached file, it is something related to file paths on Windows: Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 65: ...

@MikeUnwalla Have you tried mvn clean install? This will run tests for all languages. You can run mvn clean install -DskipTests to skip the tests.

MikeUnwalla commented 8 months ago

@jaumeortola, I used: mvn clean --projects languagetool-language-modules/en --also-make test

I have got a meeting in a couple of minutes. Let me know if you want me to run mvn clean install -DskipTests , and I will do it in a couple of hours.

jaumeortola commented 8 months ago

I would try mvn clean install, yes.

MikeUnwalla commented 8 months ago

@jaumeortola, mvn clean install -DskipTests gives a build success.

But, if I do not skip the tests (use: mvn clean install), the build fails. I attach the report: org.languagetool.DynamicMorfologikLanguageTest.txt

Also, don't forget the testrules failure. I downloaded the snapshot, unizpped it, and then got the testrules failure.

juzyz commented 8 months ago

I got the same error

'[ERROR] Errors: [ERROR] DynamicMorfologikLanguageTest.test:35 ? Runtime java.lang.RuntimeException: Could not check sentence (language: Testlang): test'

when execute command 'mvn clean test' for just cloned repository via command 'git clone --depth 5 https://github.com/languagetool-org/languagetool.git'. I used a Windows Command Prompt.

MikeUnwalla commented 8 months ago

I think that I have found the cause of the problem (for the GUI):

        ... 33 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict

Files that were in 6.3 are not in 6.4 snapshot (2024-02-20):

image

jaumeortola commented 8 months ago

Files that were in 6.3 are not in 6.4 snapshot (2024-02-20):

Those files were moved to an external dependency (english-pos-dict). From the directory where you execute java -jar languagetool.jar, they should be in /libs/english-pos-dict.jar. Can you check that this file is there? You could even unzip the file (make a copy, rename it with a zip extension, and unzip), and verify that the files are there.

MikeUnwalla commented 8 months ago

The directory/libsis not in my clone. I did not find english-pos-dict.jar with a Windows search (but Windows search almost useless).

image

jaumeortola commented 8 months ago

This file can be found in the snapshots: https://internal1.languagetool.org/snapshots/ (download a snapshot and decompress).

Locally, it can be found at languagetool/languagetool-standalone/target/LanguageTool-6.4-SNAPSHOT/LanguageTool-6.4-SNAPSHOT/ after doing mvn clean install -DskipTests.

jaumeortola commented 8 months ago

Your problem is related to file paths on Windows. Later, I will be able to check it on Windows. Maybe @fabrichter or @SteVio89 have some idea. 

Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 65: T:\GitHubClone\languagetool\languagetool-language-modules\en\file:\C:\Users\Mike\.m2\repository\org\languagetool\english-pos-dict\0.3\english-pos-dict-0.3.jar!\org\languagetool\resource\en\hunspell\en_US.dict
    at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
    at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
    at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:232)
    at java.base/java.nio.file.Path.of(Path.java:147)
    at java.base/java.nio.file.Paths.get(Paths.java:69)
    at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.initSpellers(MorfologikSpellerRule.java:226)
    at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.match(MorfologikSpellerRule.java:127)
    at org.languagetool.JLanguageTool.checkAnalyzedSentence(JLanguageTool.java:1447)
    at org.languagetool.JLanguageTool.access$1600(JLanguageTool.java:73)
    at org.languagetool.JLanguageTool$TextCheckCallable.getOtherRuleMatches(JLanguageTool.java:2051)
    ... 39 more
MikeUnwalla commented 8 months ago

The problem is not only related to file paths on Windows. There are testruleserrors with the snapshots.

Although english-pos-dict.jar is in the GUI that I made with Maven, testrules gives the error about the missing file\en\english.dict:

T:\GitHubClone\languagetool\languagetool-standalone\target\LanguageTool-6.4-SNAPSHOT\LanguageTool-6.4-SNAPSHOT>testrules en
Running XML pattern tests...
LanguageTool version 6.4-SNAPSHOT (2024-02-21 10:56:43 +0000, eb53325)
Known languages: [Arabic, Asturian, Belarusian, Breton, Catalan, Catalan (Valencian), Catalan (Balearic), Danish, German, German (Germany), German (Austria), German (Swiss), Simple German, Greek, English, English (US), English (GB), English (Australian), English (Canadian), English (New Zealand), English (South African), Esperanto, Spanish, Spanish (voseo), Persian, French, French (Canada), French (Switzerland), French (Belgium), Irish, Galician, Italian, Japanese, Khmer, Dutch, Dutch (Belgium), Polish, Portuguese, Portuguese (Portugal), Portuguese (Brazil), Portuguese (Angola preAO), Portuguese (Moçambique preAO), Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Tagalog, Ukrainian, Chinese, Crimean Tatar, Testlanguage]
Running XML validation for en/grammar.xml...
Running XML validation for en/style.xml...
Running XML validation for /org/languagetool/rules/en/remote-rule-filters.xml...
Running pattern rule tests for English (org.languagetool.language.English)...
Exception in thread "main" java.lang.RuntimeException: Could not activate rules
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:94)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:84)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:67)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:51)
        at org.languagetool.rules.patterns.PatternRuleTest.createToolForTesting(PatternRuleTest.java:210)
        at org.languagetool.rules.patterns.PatternRuleTest.runTestForLanguage(PatternRuleTest.java:192)
        at org.languagetool.rules.patterns.PatternRuleTest.runGrammarRulesFromXmlTestIgnoringLanguages(PatternRuleTest.java:184)
        at org.languagetool.rules.patterns.PatternRuleTest.main(PatternRuleTest.java:927)
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml'
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
        at org.languagetool.Language.getPatternRules(Language.java:705)
        at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
        ... 9 more
Caused by: java.lang.RuntimeException: Could not create filter class using constructor public org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter() throws java.io.IOException
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:54)
        at org.languagetool.rules.patterns.XMLRuleHandler.setRuleFilter(XMLRuleHandler.java:684)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:779)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.endElement(PatternRuleHandler.java:502)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:618)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endNamespaceScope(XMLDTDValidator.java:1985)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.handleEndElement(XMLDTDValidator.java:1934)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endElement(XMLDTDValidator.java:865)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1728)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2899)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:542)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:889)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:825)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:637)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326)
        at java.xml/javax.xml.parsers.SAXParser.parse(SAXParser.java:197)
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:76)
        ... 12 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:47)
        ... 34 more
Caused by: java.lang.ExceptionInInitializerError
        at org.languagetool.language.English.createDefaultTagger(English.java:120)
        at org.languagetool.Language.getTagger(Language.java:440)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.<init>(AbstractSuppressMisspelledSuggestionsFilter.java:39)
        at org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter.<init>(EnglishSuppressMisspelledSuggestionsFilter.java:29)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
        ... 37 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5020)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller.getDictionaryWithCaching(MorfologikSpeller.java:97)
        at org.languagetool.tagging.BaseTagger.<init>(BaseTagger.java:88)
        at org.languagetool.tagging.en.EnglishTagger.<init>(EnglishTagger.java:42)
        at org.languagetool.tagging.en.EnglishTagger.<clinit>(EnglishTagger.java:38)
        ... 42 more
Caused by: java.nio.file.NoSuchFileException: \en\english.dict
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:236)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at morfologik.stemming.Dictionary.read(Dictionary.java:63)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:57)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:50)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3571)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2313)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2190)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2080)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4012)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4035)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5011)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5018)
        ... 46 more
Running disambiguator rule tests...
Running disambiguation tests for English...
Exception in thread "main" java.lang.RuntimeException: Could not activate rules
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:338)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:244)
        at org.languagetool.tagging.disambiguation.rules.DisambiguationRuleTest.testDisambiguationRulesFromXML(DisambiguationRuleTest.java:71)
        at org.languagetool.tagging.disambiguation.rules.DisambiguationRuleTest.main(DisambiguationRuleTest.java:246)
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml'
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
        at org.languagetool.Language.getPatternRules(Language.java:705)
        at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
        ... 5 more
Caused by: java.lang.RuntimeException: Could not create filter class using constructor public org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter() throws java.io.IOException
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:54)
        at org.languagetool.rules.patterns.XMLRuleHandler.setRuleFilter(XMLRuleHandler.java:684)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:779)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.endElement(PatternRuleHandler.java:502)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:618)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endNamespaceScope(XMLDTDValidator.java:1985)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.handleEndElement(XMLDTDValidator.java:1934)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endElement(XMLDTDValidator.java:865)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1728)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2899)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:542)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:889)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:825)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:637)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326)
        at java.xml/javax.xml.parsers.SAXParser.parse(SAXParser.java:197)
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:76)
        ... 8 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:47)
        ... 30 more
Caused by: java.lang.ExceptionInInitializerError
        at org.languagetool.language.English.createDefaultTagger(English.java:120)
        at org.languagetool.Language.getTagger(Language.java:440)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.<init>(AbstractSuppressMisspelledSuggestionsFilter.java:39)
        at org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter.<init>(EnglishSuppressMisspelledSuggestionsFilter.java:29)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
        ... 33 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5020)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller.getDictionaryWithCaching(MorfologikSpeller.java:97)
        at org.languagetool.tagging.BaseTagger.<init>(BaseTagger.java:88)
        at org.languagetool.tagging.en.EnglishTagger.<init>(EnglishTagger.java:42)
        at org.languagetool.tagging.en.EnglishTagger.<clinit>(EnglishTagger.java:38)
        ... 38 more
Caused by: java.nio.file.NoSuchFileException: \en\english.dict
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:236)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at morfologik.stemming.Dictionary.read(Dictionary.java:63)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:57)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:50)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3571)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2313)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2190)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2080)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4012)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4035)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5011)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5018)
        ... 42 more
Running XML bitext pattern tests...
Bitext pattern tests successful.
Validating false-friends.xml...
Validation successfully finished.
jaumeortola commented 7 months ago

I can reproduce the error on Windows, and it can be easily solved by disabling or removing the test in en/DynamicMorfologikLanguageTest.java. Does it mean that DynamicMorfologikLanguage.java doesn't work properly on Windows? Maybe nobody uses a server with configuration on Windows. Is it okay to just disable the test? Do we need to address the problem? @fabrichter @SteVio89

MikeUnwalla commented 7 months ago

For the record, I do not use a server with configuration on Windows. Thus, the defects cannot only be related to a server with configuration on Windows. The problem with standalone LT cannot be related to a server with configutation on Windows because no server is involved (download, unzip, testrules, fail).

jaumeortola commented 7 months ago

I disabled the test here. Please check if it solves the issue for you. https://github.com/languagetool-org/languagetool/commit/b5c89420fded0fc69e26a7343c4ae546a973a95c

The problem with standalone LT cannot be related to a server with configutation on Windows because no server is involved (download, unzip, testrules, fail).

I couldn't reproduce the problem with standalone LT (downloading a snapshot).

MikeUnwalla commented 7 months ago

My version of Windows: Windows 10 Pro, version 22H2.

I synched my repository to get your change.

When I run mvn clean install, I get error messages about the command line version (as best I remember, I did not get these messages 2 days ago). Surefire report attached: org.languagetool.commandline.MainTest.txt

I ran mvn clean install -DskipTests to make a new GUI. Testrules fails: Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict

For the latest snaphot (https://internal1.languagetool.org/snapshots/LanguageTool-20240222-snapshot.zip,) testrules gives me an error message: Caused by: java.nio.file.NoSuchFileException: \en\english.dict Given that you made the change this morning, there is some weirdness going on. Why don't you get a testrules error message for the snapshot?

MikeUnwalla commented 7 months ago

@jaumeortola,

Is the testrules error caused by testrules looking for files that have been moved? (And thus, the solution is to remove the now redundant checks for the existence of the files?)

My reasoning: I downloaded and unzipped https://internal1.languagetool.org/snapshots/LanguageTool-20240225-snapshot.zip. I unzipped english-pos-dict.jar and put the files into\org\languagetool\resource\en. Except for an unrelated error, testrules is happy: 

T:<snip>\LanguageTool-6.4-SNAPSHOT>testrules en
Running XML pattern tests...
LanguageTool version 6.4-SNAPSHOT (2024-02-25 18:59:28 +0100, 77fff17)
Known languages: [Arabic, Asturian, Belarusian, Breton, Catalan, Catalan (Valencian), Catalan (Balearic), Danish, German, German (Germany), German (Austria), German (Swiss), Simple German, Greek, English, English (US), English (GB), English (Australian), English (Canadian), English (New Zealand), English (South African), Esperanto, Spanish, Spanish (voseo), Persian, French, French (Canada), French (Switzerland), French (Belgium), Irish, Galician, Italian, Japanese, Khmer, Dutch, Dutch (Belgium), Polish, Portuguese, Portuguese (Portugal), Portuguese (Brazil), Portuguese (Angola preAO), Portuguese (Moçambique preAO), Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Tagalog, Ukrainian, Chinese, Crimean Tatar, Testlanguage]
Running XML validation for en/grammar.xml...
Running XML validation for en/style.xml...
Running XML validation for /org/languagetool/rules/en/remote-rule-filters.xml...
Running pattern rule tests for English (org.languagetool.language.English)...
Check rule id uniqueness for English...
Loaded 6117 rules...
Check that sentence start tag is not included in <marker>....
Check that <unify-ignore> is not at the start of <unify>....
Check parenthesis and back references in synthesis matches...
Checking regexp syntax of 6088 rules for English...
Checking messages for 'TBD' etc of 6088 rules for English...
Checking example sentences of 6088 rules for English...
Testing rule 100...
Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error analyzing sentence: '<S> Children[child/NNS,B-NP-plural|E-NP-plural] are[be/VBP,B-VP] also[also/RB,B-ADJP] more[more/RP,many/JJR,much/JJR,I-ADJP] susceptible[susceptible/JJ,I-ADJP],[,/,,,/PCT,,/,,O] with[with/IN,with/RP,B-PP] two[two/CD,two/JJ,two/NN,B-NP-singular|E-NP-singular]-[-/null,O] to[to/IN,to/TO,B-VP] four-year-olds[four-year-old/NNS,four-year-olds/NNS,B-NP-plural|E-NP-plural] having[having/NN:UN,have/VBG,B-VP] the[the/DT,B-NP-plural] highest[high/JJS,I-NP-plural] rates[rate/NNS,rate/VBZ,E-NP-plural] of[of/IN,B-PP] infection[infection/NN:UN,B-NP-singular|E-NP-singular].[./.,</S>./PCT,<P/>,O]' with rule EN_SPLIT_WORDS_HYPHEN[1]
        at org.languagetool.rules.patterns.PatternRuleTest.testGrammarRulesFromXML(PatternRuleTest.java:549)
        at org.languagetool.rules.patterns.PatternRuleTest.runTestForLanguage(PatternRuleTest.java:202)
        at org.languagetool.rules.patterns.PatternRuleTest.runGrammarRulesFromXmlTestIgnoringLanguages(PatternRuleTest.java:184)
        at org.languagetool.rules.patterns.PatternRuleTest.main(PatternRuleTest.java:927)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error analyzing sentence: '<S> Children[child/NNS,B-NP-plural|E-NP-plural] are[be/VBP,B-VP] also[also/RB,B-ADJP] more[more/RP,many/JJR,much/JJR,I-ADJP] susceptible[susceptible/JJ,I-ADJP],[,/,,,/PCT,,/,,O] with[with/IN,with/RP,B-PP] two[two/CD,two/JJ,two/NN,B-NP-singular|E-NP-singular]-[-/null,O] to[to/IN,to/TO,B-VP] four-year-olds[four-year-old/NNS,four-year-olds/NNS,B-NP-plural|E-NP-plural] having[having/NN:UN,have/VBG,B-VP] the[the/DT,B-NP-plural] highest[high/JJS,I-NP-plural] rates[rate/NNS,rate/VBZ,E-NP-plural] of[of/IN,B-PP] infection[infection/NN:UN,B-NP-singular|E-NP-singular].[./.,</S>./PCT,<P/>,O]' with rule EN_SPLIT_WORDS_HYPHEN[1]
        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at org.languagetool.rules.patterns.PatternRuleTest.testGrammarRulesFromXML(PatternRuleTest.java:547)
        ... 3 more
Caused by: java.lang.RuntimeException: Error analyzing sentence: '<S> Children[child/NNS,B-NP-plural|E-NP-plural] are[be/VBP,B-VP] also[also/RB,B-ADJP] more[more/RP,many/JJR,much/JJR,I-ADJP] susceptible[susceptible/JJ,I-ADJP],[,/,,,/PCT,,/,,O] with[with/IN,with/RP,B-PP] two[two/CD,two/JJ,two/NN,B-NP-singular|E-NP-singular]-[-/null,O] to[to/IN,to/TO,B-VP] four-year-olds[four-year-old/NNS,four-year-olds/NNS,B-NP-plural|E-NP-plural] having[having/NN:UN,have/VBG,B-VP] the[the/DT,B-NP-plural] highest[high/JJS,I-NP-plural] rates[rate/NNS,rate/VBZ,E-NP-plural] of[of/IN,B-PP] infection[infection/NN:UN,B-NP-singular|E-NP-singular].[./.,</S>./PCT,<P/>,O]' with rule EN_SPLIT_WORDS_HYPHEN[1]
        at org.languagetool.rules.patterns.PatternRuleMatcher.match(PatternRuleMatcher.java:103)
        at org.languagetool.rules.patterns.PatternRule.match(PatternRule.java:218)
        at org.languagetool.rules.patterns.PatternRuleTest.match(PatternRuleTest.java:843)
        at org.languagetool.rules.patterns.PatternRuleTest.testCorrectSentences(PatternRuleTest.java:811)
        at org.languagetool.rules.patterns.PatternRuleTest.lambda$testGrammarRulesFromXML$2(PatternRuleTest.java:536)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException: Cannot invoke "org.languagetool.rules.spelling.morfologik.MorfologikMultiSpeller.isMisspelled(String)" because "speller" is null
        at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.isMisspelled(MorfologikSpellerRule.java:291)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.isMisspelled(MorfologikSpellerRule.java:278)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.isMisspelled(AbstractSuppressMisspelledSuggestionsFilter.java:83)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.acceptRuleMatch(AbstractSuppressMisspelledSuggestionsFilter.java:54)
        at org.languagetool.rules.patterns.RuleFilterEvaluator.runFilter(RuleFilterEvaluator.java:46)
        at org.languagetool.rules.patterns.PatternRuleMatcher.createRuleMatch(PatternRuleMatcher.java:195)
        at org.languagetool.rules.patterns.PatternRuleMatcher.lambda$match$1(PatternRuleMatcher.java:86)
        at org.languagetool.rules.patterns.AbstractPatternRulePerformer.matchFrom(AbstractPatternRulePerformer.java:161)
        at org.languagetool.rules.patterns.AbstractPatternRulePerformer.doMatch(AbstractPatternRulePerformer.java:75)
        at org.languagetool.rules.patterns.PatternRuleMatcher.match(PatternRuleMatcher.java:85)
        ... 8 more
Testing rule 200...
Testing rule 300...
<snip>
Testing rule 5700...
Testing rule 5800...
Testing rule 5900...
Running disambiguator rule tests...
Running disambiguation tests for English...
100...
200...
300...
400...
500...
600...
700...
800...
900...
1000...
1003 rules tested (3968ms)
Disambiguator tests successful.
Running XML bitext pattern tests...
Bitext pattern tests successful.
Validating false-friends.xml...
Validation successfully finished.
MikeUnwalla commented 7 months ago

@jaumeortola wrote: I couldn't reproduce the problem with standalone LT (downloading a snapshot).

With LT snapshot 2024-02-27, the problem remains: testrules fails. The disambiguation rules and the grammar rules are not checked. Thus, if I make a rule, I cannot be sure that the rule is correct.

@languagetool-org/developers, can you reproduce the testrules problem on Windows?

milekpl commented 7 months ago

@MikeUnwalla I get the error mentioned in the issue #10397 under Windows (using mvn clean test).

jaumeortola commented 7 months ago

@MikeUnwalla: Does the problem persist? I asked a couple of people to run the tests on Windows, but they couldn't reproduce the problem.

danielnaber commented 7 months ago

Could this be related to the version of Java? @milekpl which version do you use?

milekpl commented 7 months ago

@danielnaber semeru (IBM JDK version 19.0.2). I have also openjdk 21 installed but it didn't work for me. Which version should I use?

danielnaber commented 7 months ago

Could you try whether it works with Java 11?

milekpl commented 7 months ago

@danielnaber it does! mvn clean test and mvn clean install work. But testrules does NOT (I made sure it uses Java 11). Here's my stderr log;

error_log.txt

milekpl commented 7 months ago

and the file english-pos-dict.jar is found in my libs dir.

MikeUnwalla commented 7 months ago

@jaumeortola, sorry for my late reply. I was on vacation.

With snapshot 2024-03-21, I get the testrules error message.

MikeUnwalla commented 7 months ago

Java 8 is on my computer. @danielnaber wrote about Java 11.

@jaumeortola, which version of Java is necessary? 'The three-minute introduction' on https://dev.languagetool.org/development-overview tells me to download the standalone version of LT, but it does not tell me about Java.

MikeUnwalla commented 6 months ago

Fixed in LanguageTool 6.4 (https://languagetool.org/download/LanguageTool-6.4.zip). Thank you.