Debugger have unreadable output because color sequences aren't recognized

[xipion]

Describe the bug Enter debugger mode by Alt-Pause, see clubbered output, escape sequences for colouring output aren't recognized. See attached screenshot how it looks like. No special config options were enabled, it even reproduces with default .conf file or no file at all.

To Reproduce Steps to reproduce the behavior:

1. Run dosbox-x without any config
2. After seeing dos prompt, enter debugger (alt-pause)
3. See attached screenshot with unreadable output

Expected behavior Debugger output is clear. If colouring isn't support it can be monochrome, but readable.

Screenshots

Environment (please complete the following information): System: Windows 8.1 x64, dosbox-x both 32/64 bit compiled by MSVC version 0.83.1 downloaded as dosbox-x-vsbuild-win64-20210601022229.zip or dosbox-x-vsbuild-win32-20210601022229.zip

[Wengier]

It seems that the console you use does not support ANSI coloring. I don't have Windows 8.1 any more, but when I tried it on Windows 10 the correct Debugger output should be the following:

Try run DOSBox-X from the built-in "Command Prompt" of Windows instead of another console. It is hoped that the console-based debugger to be replaced with a GUI-based debugger in the future so that the debugger will be more usable as a whole (and will definitely not have such issues).

[Wengier]

I tried to run the DOSBox-X Debugger in Windows XP (with patched Visual Studio builds), and it appears to render fine too:

So it appears that the console you used for launching DOSBox-X in your Windows is not the default console, or ANSI coloring is not enabled there. With the default console (and default setting) I think the Debugger interface should show ANSI colors just fine for at least Windows XP and later.

[xipion]

Maybe it is possible to check in ANSI coloring is supported and if not, do not use it in debugger window? Or maybe ANSI coloring can be enabled/disabled in console somehow, so program should check it and enable if it not enabled yet.

I googled a little and found this: https://docs.microsoft.com/en-us/windows/console/setconsolemode So, it seems that dosbox-x should use this API to be able to output colourized ANSI output. Also seems, that my default console disables these sequences for some reason (I do not know how to switch it), so for dosbox-x it is anyway better to enable such mode before using it.

[Wengier]

@xipion Thanks for the link about the SetConsoleMode function. But I wonder where ANSI is mentioned in the page (I did not find anything about ANSI on that page yet).

[xipion]

According to this links: https://superuser.com/questions/413073/windows-console-with-ansi-colors-handling https://www.dostips.com/forum/viewtopic.php?t=9144

one can enable handling of ANSI sequences in console by issuing SetConsoleMode(..., ENABLE_VIRTUAL_TERMINAL_PROCESSING) for Windows 10 at least. I'm still googling what can be done for older Windows, like my 8.1.

[Wengier]

SetConsoleMode with ENABLE_VIRTUAL_TERMINAL_PROCESSING is already in the code below. Probably has no effort for your Windows 8.1 though.

https://github.com/joncampbell123/dosbox-x/blob/master/src/debug/debug_win32.cpp#L81

[maron2000]

Hi, may be not in your case, but using ConEmu (link), the problem is reproduceable. May be the same by using similar terminal software. Hope it helps you.

• DOSBox-X 0.83.15 pre-release version SDL1 64-bit
• Windows 8.1 Pro 64-bit
• ConEmu 210422 preview version

Here are the steps. Step 1. Confirm ANSI-settings in ConEmu (ANSI and xterm sequences to be CHECKed)

Step 2. Start DOSBox-X from ConEmu (simply typing dosbox-x will do)

Step 3. Start debugger. You will find no coloring, the sequences are visible. (Forget about the Japanese characters, it should be a box-character if code page is properly set)

The screenshot below is when ANSI and xterm sequences option is unchecked. (Restart of ConEmu may be required)

[Wengier]

@maron2000 Right, unchecking “ANSI and xterm sequences” option fixes the issue in the case when ConEmu is used (I also have it).