trickyloki3 / script-translator

a translator from script to description using templates
7 stars 6 forks source link

Utf8 chars support is broken in latest version #1

Open waken22 opened 1 year ago

waken22 commented 1 year ago

Hi, in #a609403 commit the support for utf8 chars works as expected but in latest commits it detects specific spanish characters as wrong chars:

Example: "áéíóúüñ"

This is the output error:

yaml.c (yaml_newline:169): expected newline.
yaml.c (yaml_plain:262): failed to newline yaml object.
yaml.c (yaml_map:312): failed to plain yaml object.
yaml.c (yaml_document:205): failed to map yaml object.
yaml.c (yaml_parse:106): failed to document yaml object.
pj59.c (main:60): failed to statement parse table object.

And the translation in statement.yml

- identifier: sc_start2
  print:
    - |
      [Efecto de estado]
      {*|sc_start2}
    - {4|sc_start_rate}
    - |
      Duración: {1|time/ms}

Regards

waken22 commented 1 year ago

The issue was introduced in 7e9dec7cfa50ea58b0a0a1b37c16a9c92fb9d2f5 @trickyloki3

trickyloki3 commented 1 year ago

Yeah, I removed the utf8 decoder in yaml_scanner.l because I was planning on generating my own utf8 decoder using https://github.com/trickyloki3/regular-expression, but I never got around to it. ):

You can add support for utf8 by reverting the yaml_scanner.l back to 9552d0a.

waken22 commented 1 year ago

Yeah that's what i did, maybe you should have made another branch for this kind of breaking changes :P but it's ok. I'm still figuring out how to change some formatting specifically for conditionals, because RO items descriptions are different when there is a conditional. But that's a different topic ^^

Thanks for your time.