mlua-rs / mlua

High level Lua 5.4/5.3/5.2/5.1 (including LuaJIT) and Roblox Luau bindings to Rust with async/await support
Other
1.72k stars 139 forks source link

Cannot use Chinese in lua file #449

Closed xphost008 closed 1 month ago

xphost008 commented 2 months ago

I try to use like this:

let test = mlua::Lua::new();
test.load("print(\"你好,中国!\")").exec().unwrap();

it output mistaken code in Chinese. Can you solve it?

alerque commented 2 months ago

I don't see a problem, at least testing with Lua 5.4. What Lua backend are you using?

#!/usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! mlua = { version = "0.9", features = [ "lua54" ] }
//! ```

fn main() {
    let test = mlua::Lua::new();
    test.load(r#"print("你好,中国!")"#).exec().unwrap();
}

Saved and chmod 755, then run, I get:

$ ./foo.rs
你好,中国!

Note this looks identical to the input in my terminal. What are you seeing?

Also note I used an easier way of escaping the double quotes with a raw string in Rust, but I tried it your way too and got the same result. You could also use Lua's other string delimiters, e.g.:

    test.load("print([[你好,中国!]])").exec().unwrap();
xphost008 commented 2 months ago

I don't see a problem, at least testing with Lua 5.4. What Lua backend are you using?

#!/usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! mlua = { version = "0.9", features = [ "lua54" ] }
//! ```

fn main() {
    let test = mlua::Lua::new();
    test.load(r#"print("你好,中国!")"#).exec().unwrap();
}

Saved and chmod 755, then run, I get:

$ ./foo.rs
你好,中国!

Note this looks identical to the input in my terminal. What are you seeing?

Also note I used an easier way of escaping the double quotes with a raw string in Rust, but I tried it your way too and got the same result. You could also use Lua's other string delimiters, e.g.:

    test.load("print([[你好,中国!]])").exec().unwrap();

I use lua54, it appear like this:

let test = mlua::Lua::new();
test.load(r#"print("你好,中国!")").exec().unwrap();

output is:

cargo run
浣犲ソ锛屼腑鍥斤紒

Perhaps GBK was used internally to read UTF-8 encoding, at least I read it from this URL Pass in the output, then find 'Original UTF-8 Used' below and read 'Incorrect GBK Used'..

Please tell me how to use UTF-8 in mlua btw, I use Windows 11 24H2 version, because I haven't Linux environment.

eigeen commented 2 months ago

I tried to run lua ./test.lua in the default powershell environment, and it outputs wrong characters as same as in mlua. It's not just mlua's problem.

xphost008 commented 2 months ago

I tried to run lua ./test.lua in the default powershell environment, and it outputs wrong characters as same as in mlua. It's not just mlua's problem.

but... I use lua ./test.lua if my file encoding is GBK, that output is normal, but I change encoding to UTF-8, that output wrong.. look below picture: image the up output is UTF-8 encoding, the below output is GBK. but I use mlua in Rust, however I use any encoding, it just output wrong.. so, that not my fault!

xphost008 commented 2 months ago

I use mlua in RustRover, You can see my source: This is get_file function: image This is my load_plugin function: image You can watch it! thank you answer!

eigeen commented 2 months ago

Because the String encoding in Rust is always UTF-8, it is not affected by the source file encoding. You should set your terminal output encoding to UTF-8 in any way.

Active code page: 65001

C:\dev\mlua-test>cargo run  
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.05s
     Running `target\debug\mlua-test.exe`
你好,中国!
xphost008 commented 2 months ago

Because the String encoding in Rust is always UTF-8, it is not affected by the source file encoding. You should set your terminal output encoding to UTF-8 in any way.

Active code page: 65001

C:\dev\mlua-test>cargo run  
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.05s
     Running `target\debug\mlua-test.exe`
你好,中国!

Ha, whatever i run my rust file, I just need to run chcp 65001 at first? that sounds not good, my users think my software be Chinese support is not good.. However thank you! at least I know how to fix my software!

eigeen commented 2 months ago

Because the String encoding in Rust is always UTF-8, it is not affected by the source file encoding. You should set your terminal output encoding to UTF-8 in any way.

Active code page: 65001

C:\dev\mlua-test>cargo run  
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.05s
     Running `target\debug\mlua-test.exe`
你好,中国!

Ha, whatever i run my rust file, I just need to run chcp 65001 at first? that sounds not good, my users think my software be Chinese support is not good.. However thank you! at least I know how to fix my software!

如果你软件使用mlua是为了嵌入脚本,尝试覆盖print函数,使用其他方式输出,例如log::info!或者println!,这应该是最合适的方案。mlua的实现可能是直接使用了lua库默认行为,才导致和直接运行lua的行为一样,源文件编码与控制台不匹配就会乱码。

eigeen commented 2 months ago

Here is an example for you to override the original print function:

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let test = mlua::Lua::new();
    test.globals().set(
        "print",
        test.create_function(|_, values: mlua::Variadic<String>| {
            println!("{}", values.join(", "));
            Ok(())
        })?,
    )?;

    test.load(r#"print("你好,中国!", 123, 456.789)"#).exec()?;

    Ok(())
}

Note that it's a minimal example, if you want print behave like the original lua, you'll need more work to handle the various types, by accepting mlua::Variadic<mlua::Value>.

xphost008 commented 2 months ago

Here is an example for you to override the original print function:

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let test = mlua::Lua::new();
    test.globals().set(
        "print",
        test.create_function(|_, values: mlua::Variadic<String>| {
            println!("{}", values.join(", "));
            Ok(())
        })?,
    )?;

    test.load(r#"print("你好,中国!", 123, 456.789)"#).exec()?;

    Ok(())
}

Note that it's a minimal example, if you want print behave like the original lua, you'll need more work to handle the various types, by accepting mlua::Variadic<mlua::Value>.

ok thanks! I know how to repair it!