XAMPPRocky / fluent-templates

Easily add Fluent to your Rust project.
Apache License 2.0
136 stars 28 forks source link

Unicode Symbols #72

Closed Fedox-die-Ente closed 2 weeks ago

Fedox-die-Ente commented 1 month ago

Hey, im currently testing out the crate.

This is my code:

use std::borrow::Cow;
use std::collections::HashMap;
use fluent_templates::{LanguageIdentifier, Loader, static_loader};
use fluent_templates::fluent_bundle::{FluentArgs, FluentValue};
use std::string::String;
use fluent_templates::fs::langid;

const US_ENGLISH: LanguageIdentifier = langid!("en-US");

static_loader! {
    static LOCALES = {
        locales: "locales",
        fallback_language: "en-US",
    };
}
fn main() {

    let mut args: HashMap<String, FluentValue> = HashMap::new();
    args.insert("name".to_string(), FluentValue::String(Cow::from("John")));

    println!("Debug output: {:?}", l(US_ENGLISH, "test2", &args));
    println!("{}", l(US_ENGLISH, "test2", &args));
}

pub fn l(language_identifier: LanguageIdentifier, text_id: &str, fluent_args: &HashMap<String, FluentValue>) -> String {
    if !fluent_args.is_empty() {
        LOCALES.lookup_with_args(&language_identifier, text_id, &fluent_args)
    } else {
        LOCALES.lookup(&language_identifier, text_id)
    }
}

And this is the output: image

Someone know why this happens?

alerque commented 1 month ago

This is not a full MWE for your situation, so it is not possible to debug. You need to also post the FTL data and anything else this needs to run (and whene to put it to run). I suggest using something like a stand alone script to test with. For example with rust-script you could do something like this:

#!/usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! fluent-templates = "0.9"
//! ```

use fluent_templates::fluent_bundle::{FluentArgs, FluentValue};
use fluent_templates::fs::langid;
use fluent_templates::{static_loader, LanguageIdentifier, Loader};
use std::borrow::Cow;
use std::collections::HashMap;
use std::string::String;

const US_ENGLISH: LanguageIdentifier = langid!("en-US");

static_loader! {
    static LOCALES = {
        locales: "locales",
        fallback_language: "en-US",
    };
}

fn main() {
    let mut args: HashMap<String, FluentValue> = HashMap::new();
    args.insert("name".to_string(), FluentValue::String(Cow::from("John")));

    println!("Debug output: {:?}", l(US_ENGLISH, "test2", &args));
    println!("{}", l(US_ENGLISH, "test2", &args));
}

pub fn l(
    language_identifier: LanguageIdentifier,
    text_id: &str,
    fluent_args: &HashMap<String, FluentValue>,
) -> String {
    if !fluent_args.is_empty() {
        LOCALES.lookup_with_args(&language_identifier, text_id, &fluent_args)
    } else {
        LOCALES.lookup(&language_identifier, text_id)
    }
}

... but that doesn't actually run yet because not everything is there yet. In particular it would be nice for the sake of an MWE to either have a known FTL file and location to put it relative to the script or just inline the data somehow for testing.

zbraniecki commented 1 month ago

Hi, this is to be expected. Fluent isolates placeables in what's called Bidirectional-Isolation Unicode characters allowing for Left-To-Right and Right-To-Left mixing. (think - English name in Arabic translation).

https://github.com/projectfluent/fluent.js/wiki/Unicode-Isolation

alerque commented 1 month ago

Hmm, this actually explain a bug I couldn't make out a while back with translated strings with filenames that could not be copy-pasted from the terminal because they were dragging these isolation characters with them. At the time I didn't have info from the user about what the character it was, only that the copied data worked some places not others.

I see in FluentBundle this can be disabled with bundle.set_use_isolating(false);. Is there a way to reach that option in fluent-templates?

XAMPPRocky commented 2 weeks ago

Thank you for your issue! This is covered by the FAQ in the readme.

https://github.com/XAMPPRocky/fluent-templates?tab=readme-ov-file#faq