Output of Jansi is corrupted in cygwin console on Windows7.

[tyan0]

Overview

In cygwin console on Windows7, output of jansi v2.1.1 is corrupted. This also happen with current git head of jansi.

Description

Output of Maven, whose jansi library is replaced with jansi v2.1.1, is corrupted in cygwin console (command prompt) on Windows7.

This does not occur in the systems which support ENABLE_VIRTUAL_TERMINAL_PROCESSING. Also works without the problem in mintty.

The Cause

Even in cygwin, ANSI escape sequence is not supported in console (command prompt) if the process is not a cygwin process. If console supports ENABLE_VIRTUAL_TERMINAL_PROCESSING, ANSI escape sequences can be used with this flag set. Windows7 does not have this flag, therefore, AnsiType.Emulation should be selected in AnsiConsole.java. However, current code decides to use ANSI escape sequences for cygwin console on Windows7.

Moreover, "ESC[0m" is sent unconditionally in AnsiOutputStream.java. This also causes garbled output in cygwin console on Windows7.

Patch for this issue

diff --git a/src/main/java/org/fusesource/jansi/AnsiConsole.java b/src/main/java/org/fusesource/jansi/AnsiConsole.java
index fb45f9f..f5c7a29 100644
--- a/src/main/java/org/fusesource/jansi/AnsiConsole.java
+++ b/src/main/java/org/fusesource/jansi/AnsiConsole.java
@@ -263,7 +263,7 @@ public class AnsiConsole {
                     }
                 };
             }
-            else if (IS_CONEMU || IS_CYGWIN || IS_MSYSTEM) {
+            else if (IS_CONEMU || ((IS_CYGWIN || IS_MSYSTEM) && GetConsoleMode(console, mode) == 0)) {
                 // ANSI-enabled ConEmu, Cygwin or MSYS(2) on Windows...
                 processor = null;
                 type = AnsiType.Native;
diff --git a/src/main/java/org/fusesource/jansi/io/AnsiOutputStream.java b/src/main/java/org/fusesource/jansi/io/AnsiOutputStream.java
index 02b86bc..b4d17d9 100644
--- a/src/main/java/org/fusesource/jansi/io/AnsiOutputStream.java
+++ b/src/main/java/org/fusesource/jansi/io/AnsiOutputStream.java
@@ -24,6 +24,8 @@ import java.util.ArrayList;
 import org.fusesource.jansi.AnsiColors;
 import org.fusesource.jansi.AnsiMode;
 import org.fusesource.jansi.AnsiType;
+import org.fusesource.jansi.Ansi;
+import org.fusesource.jansi.AnsiConsole;

 /**
  * A ANSI print stream extracts ANSI escape codes written to 
@@ -39,8 +41,6 @@ import org.fusesource.jansi.AnsiType;
  */
 public class AnsiOutputStream extends FilterOutputStream {

-    public static final byte[] RESET_CODE = "\033[0m".getBytes();
-
     public interface IoRunnable {
         void run() throws IOException;
     }
@@ -317,8 +317,7 @@ public class AnsiOutputStream extends FilterOutputStream {
                 && type != AnsiType.Redirected
                 && type != AnsiType.Unsupported) {
             setMode(AnsiMode.Default);
-            write(RESET_CODE);
-            flush();
+            AnsiConsole.out().print(Ansi.ansi().reset());
         }
         if (uninstaller != null) {
             uninstaller.run();

[gnodet]

@tyan0 Thx for the detailed analysis. I don't have a Windows 7 at hand, so I haven't been able to test it, but I slightly modified your patch. Let me know if this works for you asap, as I'd like to get a new release out soon.

[tyan0]

Thanks for fixing the problem. I pulled the git head and have confirmed that the problem disappeared in Windows 7.

[tyan0]

Sorry, my test might not be enough.

I have a question. https://github.com/fusesource/jansi/blob/fc2f4b4c3f19acc6e7ce6f17257a1e1aaa0260d4/src/main/java/org/fusesource/jansi/AnsiConsole.java#L410 Shouldn't this be as follows?

        boolean resetAtUninstall = type != AnsiType.Emulation && !getBoolean(JANSI_NORESET);

Current code seems to have no effect for Win7 console.