coursier / coursier

Pure Scala Artifact Fetching
https://get-coursier.io
Apache License 2.0
2.04k stars 310 forks source link

Multiple things about Coursier launcher completely broken on UTF-8-enabled Windows #3164

Open s5bug opened 5 days ago

s5bug commented 5 days ago

Argument UTF-8 bytes are mistakenly interpreted as Windows-1252:

> scala-cli run .\hello.scala -- 渝
Compiling project (Scala 3.5.2, JVM (23))
Compiled project (Scala 3.5.2, JVM (23))
Hello, �!

In some scenarios, i.e. when running the auxiliary executable directly, the bytes seem to be interpreted as Windows-1252, then as some form of Extended ASCII (? appears instead of when you ask windows for an A-string with invalid characters):

> .cs.aux.exe install 渝
https://repo1.maven.org/maven2/io/get-coursier/apps/maven-metadata.xml
  100.0% [##########] 3.3 KiB (130.9 KiB / s)
https://repo1.maven.org/maven2/io/get-coursier/apps/maven-metadata.xml
  No new update since 2024-11-07 11:27:17
https://repo1.maven.org/maven2/io/get-coursier/apps/1.0.58/apps-1.0.58.jar
  100.0% [##########] 11.7 KiB (532.0 KiB / s)
Cannot find app �? in channels io.get-coursier:apps

Using the installer set-up shows this off a bit more prominently: when using a Unicode user directory, the new PATH variables are created with ?s: image And so are launcher scripts: image

s5bug commented 5 days ago

At first I thought the JNI could be a problem. However on a UTF-8 enabled machine, image the "non-Unicode" ANSI functions do work properly with Unicode characters:

PS > Add-Type -TypeDefinition @"
using System;
using System.Diagnostics;
using System.Runtime.InteropServices;

public static class FileApi {
    [DllImport("Kernel32.dll", CharSet=CharSet.Ansi)]
    public static extern bool CreateDirectoryA(
        string pathName,
        nuint securityAttributes);
}
"@
PS > mkdir newEmptyDir; cd .\newEmptyDir\
PS newEmptyDir> [FileApi]::CreateDirectoryA("newEmptyDir/渝", 0)
True
PS newEmptyDir> ls
Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----          2024-11-23    09:55                渝

so I don't know where all of this stuff could be coming from.