non-ascii filenames cause error listing files

[jtippett]

Hi there,

I was testing out running a server (MacOS, build from source) and I noticed an issue. Any unicode won't list and produces this log:

2024-06-08T20:28:52.886+0700    debug   Received Transaction    {"remoteAddr": "127.0.0.1:49556", "login": "guest", "name": "guest", "RequestType": "TranGetFileNameList"}
2024-06-08T20:28:52.886+0700    error   Error handling transaction  {"remoteAddr": "127.0.0.1:49556", "login": "guest", "name": "guest", "err": "encoding: rune not supported by encoding."}

my test file names were ベスト.txt and helló.txt. Client was https://github.com/mierau/hotline.

Not sure if this is a bug in the server or client or even an issue in the protocol itself? Either way, thought I'd report it. Thanks for the cool project.

[jhalter]

Hi there. Thanks for raising this.

The Mobius project targets compatibility with the original Macintosh Hotline clients, which limits the characters that it can use to those in the Mac OS Roman encoding.

Mobius supports encoding translation for a small subset of unicode characters that have equivalents in the Mac OS Roman character set. For example, ∞ or ƒ. Not sure why ó doesn't work -- seems like it should! The Japanese characters would definitely not work though.

I'll fix the file listing transaction to log a better error and skip files with incompatible names instead of totally failing.

[tjohnman]

Historically, Japanese servers used the Shift JIS encoding. Over the years I've seen servers use Mac OS Roman, Latin 1, Shift JIS and UTF-8. But as far as I know, other than the user selecting what encoding they want to use, or the clients trying to autodetect it, the protocol never took into consideration how to handle it.

[zquestz]

I would love to see a way to start the server with the encoding of your choice, or add auto detection logic. It can still default to Mac OS Roman, but giving more flexibility is a good thing. Most people use UTF-8 these days, and it would be nice for server operators to opt into a more widely used character set.