search

languagetool-org / languagetool

Style and Grammar Checker for 25+ Languages
https://languagetool.org
GNU Lesser General Public License v2.1
12.26k stars 1.38k forks source link

Tests for v4.5.1 fail for org.languagetool.commandline.MainTest #1498

Closed Erikvl87 closed 5 years ago

Erikvl87 commented 5 years ago

For a Docker image I am experimenting with testing the LanguageTool release before publishing an actual Docker image. I've found that running the tests (mvn clean test) for the v4.5.1 release returned some errors which in turn prevents me releasing the Docker image:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.languagetool.commandline.MainTest
[ERROR] Tests run: 42, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 53.079 s <<< FAILURE! - in org.languagetool.commandline.MainTest
[ERROR] testPolishStdInDefaultOff(org.languagetool.commandline.MainTest)  Time elapsed: 0.577 s  <<< FAILURE!
java.lang.AssertionError
\tat org.junit.Assert.fail(Assert.java:86)
\tat org.junit.Assert.assertTrue(Assert.java:41)
\tat org.junit.Assert.assertTrue(Assert.java:52)
\tat org.languagetool.commandline.MainTest.testPolishStdInDefaultOff(MainTest.java:354)
\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
\tat java.lang.reflect.Method.invoke(Method.java:498)
\tat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
\tat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
\tat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
\tat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
\tat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
\tat org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
\tat org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
\tat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
\tat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
\tat org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
\tat org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
\tat org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
\tat org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
\tat org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
\tat org.junit.runners.ParentRunner.run(ParentRunner.java:363)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
\tat org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
\tat org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
\tat org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
\tat org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

[ERROR] testPolishApiStdInDefaultOff(org.languagetool.commandline.MainTest)  Time elapsed: 0.244 s  <<< FAILURE!
java.lang.AssertionError: 

Expected: is <1>
     but: was <2>
\tat org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
\tat org.junit.Assert.assertThat(Assert.java:956)
\tat org.junit.Assert.assertThat(Assert.java:923)
\tat org.languagetool.commandline.MainTest.testPolishApiStdInDefaultOff(MainTest.java:364)
\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
\tat java.lang.reflect.Method.invoke(Method.java:498)
\tat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
\tat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
\tat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
\tat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
\tat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
\tat org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
\tat org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
\tat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
\tat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
\tat org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
\tat org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
\tat org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
\tat org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
\tat org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
\tat org.junit.runners.ParentRunner.run(ParentRunner.java:363)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
\tat org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
\tat org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
\tat org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
\tat org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

[ERROR] testStdInWithExternalFalseFriends(org.languagetool.commandline.MainTest)  Time elapsed: 0.663 s  <<< FAILURE!
java.lang.AssertionError
\tat org.junit.Assert.fail(Assert.java:86)
\tat org.junit.Assert.assertTrue(Assert.java:41)
\tat org.junit.Assert.assertTrue(Assert.java:52)
\tat org.languagetool.commandline.MainTest.testStdInWithExternalFalseFriends(MainTest.java:218)
\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
\tat java.lang.reflect.Method.invoke(Method.java:498)
\tat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
\tat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
\tat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
\tat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
\tat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
\tat org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
\tat org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
\tat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
\tat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
\tat org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
\tat org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
\tat org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
\tat org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
\tat org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
\tat org.junit.runners.ParentRunner.run(ParentRunner.java:363)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
\tat org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
\tat org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
\tat org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
\tat org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
\tat org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

[INFO] Running org.languagetool.commandline.CommandLineParserTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.languagetool.commandline.CommandLineParserTest
[INFO] Running org.languagetool.commandline.CommandLineToolsTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.132 s - in org.languagetool.commandline.CommandLineToolsTest
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   MainTest.testPolishApiStdInDefaultOff:364 
Expected: is <1>
     but: was <2>
[ERROR]   MainTest.testPolishStdInDefaultOff:354
[ERROR]   MainTest.testStdInWithExternalFalseFriends:218
[INFO] 
[ERROR] Tests run: 46, Failures: 3, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] languagetool-parent ................................ SUCCESS [  3.639 s]
[INFO] LanguageTool Style and Grammar Checker Core ........ SUCCESS [ 53.732 s]
[INFO] English module for LanguageTool .................... SUCCESS [04:46 min]
[INFO] Persian module for LanguageTool .................... SUCCESS [  7.908 s]
[INFO] French module for LanguageTool ..................... SUCCESS [ 42.621 s]
[INFO] German module for LanguageTool ..................... SUCCESS [03:50 min]
[INFO] Polish module for LanguageTool ..................... SUCCESS [02:33 min]
[INFO] Catalan module for LanguageTool .................... SUCCESS [04:46 min]
[INFO] Italian module for LanguageTool .................... SUCCESS [  8.794 s]
[INFO] Breton module for LanguageTool ..................... SUCCESS [ 12.568 s]
[INFO] Dutch module for LanguageTool ...................... SUCCESS [01:03 min]
[INFO] Portuguese module for LanguageTool ................. SUCCESS [02:16 min]
[INFO] Russian module for LanguageTool .................... SUCCESS [ 38.455 s]
[INFO] Asturian module for LanguageTool ................... SUCCESS [  6.399 s]
[INFO] Belarusian module for LanguageTool ................. SUCCESS [  4.001 s]
[INFO] Chinese module for LanguageTool .................... SUCCESS [ 19.261 s]
[INFO] Danish module for LanguageTool ..................... SUCCESS [  6.346 s]
[INFO] Esperanto module for LanguageTool .................. SUCCESS [ 15.117 s]
[INFO] Galician module for LanguageTool ................... SUCCESS [ 16.737 s]
[INFO] Greek module for LanguageTool ...................... SUCCESS [  5.161 s]
[INFO] Icelandic module for LanguageTool .................. SUCCESS [  4.075 s]
[INFO] Japanese module for LanguageTool ................... SUCCESS [  8.745 s]
[INFO] Khmer module for LanguageTool ...................... SUCCESS [  5.422 s]
[INFO] Lithuanian module for LanguageTool ................. SUCCESS [  4.809 s]
[INFO] Malayalam module for LanguageTool .................. SUCCESS [  5.748 s]
[INFO] Romanian module for LanguageTool ................... SUCCESS [ 13.850 s]
[INFO] Slovak module for LanguageTool ..................... SUCCESS [ 12.664 s]
[INFO] Slovenian module for LanguageTool .................. SUCCESS [  5.821 s]
[INFO] Spanish module for LanguageTool .................... SUCCESS [ 12.179 s]
[INFO] Swedish module for LanguageTool .................... SUCCESS [  6.502 s]
[INFO] Tamil module for LanguageTool ...................... SUCCESS [  6.037 s]
[INFO] Tagalog module for LanguageTool .................... SUCCESS [  5.795 s]
[INFO] Ukrainian module for LanguageTool .................. SUCCESS [01:02 min]
[INFO] Simple German module for LanguageTool .............. SUCCESS [ 14.697 s]
[INFO] All language modules for LanguageTool .............. SUCCESS [  0.122 s]
[INFO] LanguageTool common GUI classes .................... SUCCESS [  2.751 s]
[INFO] LanguageTool command-line version .................. FAILURE [ 56.039 s]
[INFO] LanguageTool embedded HTTP server .................. SKIPPED
[INFO] LanguageTool tools for building dictionaries ....... SKIPPED
[INFO] LanguageTool stand-alone GUI ....................... SKIPPED
[INFO] LanguageTool LibreOffice/OpenOffice extension ...... SKIPPED
[INFO] Hunspell native libs for LanguageTool .............. SKIPPED
[INFO] LanguageTool Wikipedia tools ....................... SKIPPED
[INFO] LanguageTool HTTP API client ....................... SKIPPED
[INFO] LanguageTool development tools ..................... SKIPPED
[INFO] LanguageTool RPM ................................... SKIPPED

Haven't the tests been run before releasing v4.5.1 or have I been doing something wrong in my setup?

danielnaber commented 5 years ago

Did this happen once or is it reproducible? Which version of Java and which OS is this? I cannot reproduce the issue with Java 1.8.0_192 on Ubuntu 16.04.

Erikvl87 commented 5 years ago

Hi @danielnaber! Yes, I am able to reproduce this within Docker. The Docker image is running on debian:stretch (Debian 9.8) with openjdk-8-jdk-headless installed. 

openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-8u212-b01-1~deb9u1-b01)
OpenJDK 64-Bit Server VM (build 25.212-b01, mixed mode)

You'll be able to view the system configuration in my Dockerfile. I am not sure if you are known with Docker but you can reproduce this by building the mentioned Dockerfile and replacing this line: RUN ["bash", "build.sh", "languagetool-standalone", "package", "-DskipTests"] with RUN ["mvn", "clean", "test"]

danielnaber commented 5 years ago

These are all lines that check STDOUT, by redirecting it to Java (System.setOut(new PrintStream(this.out));). There's something special about STDOUT with docker, isn't it? Maybe our redirect just doesn't work and the STDOUT ends up somewhere else. I don't have time now to reproduce this now, though.

Erikvl87 commented 5 years ago

Alright, nevertheless thank you for your time. You gave me some helpful information where to look at. I'll try and do some more research myself.

Erikvl87 commented 5 years ago

I resolved the issue. Leaving the solution here for others.

It didn't had anything to do with the redirect of the STDOUT. The logs above contain stuff like Expected: is <1> but: was <2> which gave me an indication that redirects are in fact working.

Viewing the source of the failing tests made me discover that the failing tests are writing Polish characters to the input stream so I guessed it could maybe be related to character encoding. So I've installed locales and configured UTF-8 and that fixed the tests. 😃

ogryniuk commented 3 years ago

@Erikvl87 and @danielnaber,

I do not use Docker and face the very same issue when I build through 'maven clean test'. In my opinion, the issue deserves to be re-opened.

For information purposes, I run on Windows and use Java 8. My individual parameters, though, do not matter much because other people (such as @Erikvl87) also have this issue and so it most likely has a common nature.