Unicode UTF-8 Characters are displayed wrong when connecting over SSH from ArcaOS

[explit7]

Hello guys. I have a very strange problem:

Licensed ArcaOS 5.0.6.7 OS. Locale set to en_US Installed openssh rpm Package. Connecting to a linux Machine works, but UTF-8 Characters are not displayed, or displayed wrong.

See attached Midnight Commander Photo to demonstrate the problem.

Do you have some ideas?

Thanks

Also opened an ArcaOS Suport Ticket, but they suggest me to open Github issue. https://mantis.arcanoae.com/view.php?id=3098

To Reproduce Steps to reproduce the behavior:

1. Under ArcaOS open command shell and install openssh.i686 package with yum.
2. Try to establish a connection to another Linux machine.
3. After connection is established and yu open on the remote side some programs like MC (Midnight commander), you will see, that many symbols are broken.
4. On the local Linux Machine anything works fine.

Expected behavior The symbols of MC should be properly visible.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: ArcaOS 5.0.6.7
• OpenSSH_7.7p1, OpenSSL 1.0.2u 20 Dec 2019
• LANGUAGE=en_US
• TCP_LANGUAGE=en_US
  • Relevant config.sys variables:

REM MC SET MC_LIBDIR=D:/usr/lib/mc SET TERMCAP=%MC_LIBDIR%/termcap SET TMP=D:\var\tmp

REM EMX SET TERM=ansi SET TERMCAP=D:/usr/X11R6/lib/X11/etc/xterm.termcap SET EMXOPT=-h512 SET HELPNDX=EMXBOOK.NDX SET EMXBOOK=EMXDEV.INF+EMXLIB.INF+EMXGNU.INF+EMXBSD.INF SET INFOPATH=D:/usr/info

REM MANUAL SET PAGER=less REM SET MAN_CONF=D:/etc/man_db.conf REM SET MANPAGER=less REM SET MANPATH=D:/usr/share/man;D:/usr/X11R6/man

REM LOCALE SET LOCALEDIR=D:/usr/share/locale;D:/usr/X11R6/lib/X11/locale SET GNULOCALEDIR=D:/usr/share/locale SET LANGUAGE=en SET TCPLANG=en_US SET LC_CTYPE=en_US.UTF-8 SET LC_ALL=en_US.UTF-8

REM AWK SET AWK=D:/usr/bin/gawk.exe SET AWKPATH=D:/usr/share/awk

Asus-S300N-20210309-generic.log config.sys.log ssh_config.log sshd_config.log

Additional context Add any other context about the problem here.

[SilvanScherrer]

please update to latest openssh from exp repo. and retest again.

[explit7]

I enabled the netlabs-exp repo and updated alls SSH-packages.

Now ssh -V show me OpenSSH_8.4p1, OpenSSL 1.1.1h 22 Sep 2020

but the problem is still not solved and now after closing ssh i get additional error:

[SilvanScherrer]

Did you update libc libcx and pthread as well? If not please do so. As the crash should not be

[explit7]

Yes, i updated also all libc and libcx packages. The main Problem is still there The Killed by SOGSEGV Problem always occurs when i leave the remote ssh shell on a linux machine with exit command.

Also get D:\USR\BIN\bash.exe [296: 1 (255)] tcsetattr: Invalid argument

When starting ssh from bash and exit the remote shell.

Maybe some of my kLIBC User Management settings are wrong or incomplete?

Thanks

[SilvanScherrer]

The tcsetattr is not really an issue. The crash i need to reproduce first. When i have time is uncertain right now. The codepage issue as well.

[explit7]

Thanks SilvanScherrer for taking time

[explit7]

005f_01.trp.log

Exception Report - created 2021/03/09 18:01:51

---

OS2/eCS Version: 2.45 Number of Processors: 1 Physical Memory: 2014 mb Virt Addr Limit: 1536 mb Exceptq Version: 7.11.3-shl (Jul 5 2016)

---

Exception C0000005 - Access Violation

---

Process: D:\USR\BIN\SSH.EXE (02/01/2021 18:16:16 404,770) PID: 5F (95) TID: 01 (1) Priority: 200

Filename: D:\USR\LIB\PTHR01.DLL (08/12/2020 15:56:12 7,545) Address: 005B:1E350CBA (0001:00000CBA) Cause: Attempted to read from 00000000 (not a valid address)

Its a log about a trap. Maybe its somehow useful

[SilvanScherrer]

Are you sure you use latest pthread libc and libcx from exp? As timestamp of the dll say something else. And please reboot afterwards

[explit7]

Hm, i updated all packages with yum update. But will look into it. Rebooted several times. Thanks for the info

[explit7]

So, i made on clear installation of ArcaOS on the Drive C (Last installation was drive D). Updated openssh and all other packages to the newest versions from netlabs-exp repo. Now the trap after i exit ssh session is gone, but still the same unicode Problem.

As i can see the magic happens somehow inside the parameter TERM= in the CONFIG.SYS

Default Parameter after ArcaOS Installation is TERM=os2

When TERM=os2, i can SSH to the linux machine, but when starting mc on remote machine i get:

Unknown terminal: os2 Check the TERM environment variable. Also make sure that the terminal is defined in the terminfo database, Alternatively, set the TERMCAP environment variable to the desired termcap entry.

When i use TERM=ansi I got no errors, but i got broken codepage like on the photo.

When i use TERM=vt220 or TERM=xterm codepage is broken as well

[SilvanScherrer]

And with term=os2 the codepage is ok?

[explit7]

Unfortunately not. /usr/share/terminfo/o/os2 was missing on the remote Linux machine. i copied this file from the ArcaOS computer to the linux machine. Now the error about Unknown terminal: os2 is gone, but the codepage is still broken. its really weird, since i have no more ideas, what else i can change to start midnight commander on the remote ssh shell.

I think this is exact the Problem i have: https://midnight-commander.org/ticket/2339

[dryeo]

The problem is a codepage mismatch. Linux uses IBM-1208, better known as UTF8 and OS/2 is likely using codepage 850 and the characters above 127 don't match. The strings would need converting with something like UniStrFromUcs() or the more complicated UniUconvFromUcs() and the corresponding To versions.

[dmik]

If it's the utf8<->local translation issue indeed, there is no need to use the Uni API directly — there is POSIX iconv API supported by our libc (and by a separate iconv library as well, libiconv in RPM). It won't surprise me if SSH already makes use of it on other platforms.