Can't parse execute_with_pipe stdio data

[CsloudX]

Tested versions

v4.4.dev2.mono.official [97ef3c837]

System information

Godot v4.4.dev2.mono - Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1060 5GB (NVIDIA; 31.0.15.3623) - AMD Ryzen 9 3900X 12-Core Processor (24 Threads)

Issue description

I'm try get execute_with_pipe's output.

extends Control

var pipe : FileAccess
var thread : Thread

func _ready() -> void:
    var dic = OS.execute_with_pipe("cmd",["/c", "ping 127.0.0.1"])
    pipe = dic.stdio
    prints(pipe)
    thread = Thread.new()
    thread.start(_thread_func)
    get_window().close_requested.connect(clean_func)

func _thread_func():
    while pipe.is_open() and pipe.get_error()==OK:
        _add_char.call_deferred(pipe.get_8())
    prints("all done", pipe.is_open(), pipe.get_error())

var data := PackedByteArray()
func _add_char(c):
    data.append(c)
    $TextEdit.text = data.get_string_from_utf8()
    prints("ccc",c,Engine.get_frames_drawn())

func clean_func():
    pipe.close()
    thread.wait_to_finish()

func _on_line_edit_text_submitted(new_text: String) -> void:
    var cmd :String = new_text+"\n"
    var buffer = cmd.to_utf8_buffer()
    pipe.store_buffer(buffer)
    $LineEdit.clear()

but I can't get correct data.

Steps to reproduce

/

Minimal reproduction project (MRP)

CmdTest.zip

[AThousandShips]

Can't replicate on Windows 11 4.4.dev[df29cc696]

[CsloudX]

NOTE: the ascii characters was OK, need test for no ascii characters like Chinese. for the test project, it should parse to like this:

[AThousandShips]

Then I unfortunately can't test it

But do test with data.get_string_from_utf16() instead, it might be that the pipe is raw and on Windows the terminal uses utf16, not utf8

[bruvzg]

Seems like your system is set to use GB-18030 encoding for console, Godot only support UTF-8, UTF-16, UTF-32 and ASCII/Latin-1.

Try setting "cmd" to UTF-8 something like cmd /c chcp 65001 && ping 127.0.0.1 might work.

[CsloudX]

ADDTIONS:

1. but everything was OK when use OS.execute.
2. use cmd /c chcp 65001 && ping 127.0.0.1, the result changed to english

[bruvzg]

execute attempt to convert it from current Windows encoding if UTF-8 parsing fail using MultiByteToWideChar(CP_ACP, ...), pipe do not, and it will make no sense to do so since piped data is not necessarily text and should not be modified. I guess we can expose this string conversion as a new OS method.

[CsloudX]

IMO, the execute and execute_with_pipe should have same behavior and have method convert stdio string to String without chcp 65001.

[bruvzg]

I guess we can expose this string conversion as a new OS method.

https://github.com/godotengine/godot/pull/96950