search

mquinson / po4a

Maintain the translations of your documentation with ease (PO for anything)
http://po4a.org/
GNU General Public License v2.0
121 stars 58 forks source link

Yaml module fails to generate array like string #439

Closed gemmaro closed 3 weeks ago

gemmaro commented 10 months ago

Array like string literal like [ bbb ] is generated as [ bbb ], not as '[ bbb ]'. Here is a test case:

```diff Subject: [PATCH 1/2] Add array like string test case --- t/fmt-yaml.t | 4 ++++ t/fmt/yaml/arraylikestring.norm | 2 ++ t/fmt/yaml/arraylikestring.po | 23 +++++++++++++++++++++++ t/fmt/yaml/arraylikestring.pot | 23 +++++++++++++++++++++++ t/fmt/yaml/arraylikestring.trans | 2 ++ t/fmt/yaml/arraylikestring.yaml | 1 + 6 files changed, 55 insertions(+) create mode 100644 t/fmt/yaml/arraylikestring.norm create mode 100644 t/fmt/yaml/arraylikestring.po create mode 100644 t/fmt/yaml/arraylikestring.pot create mode 100644 t/fmt/yaml/arraylikestring.trans create mode 100644 t/fmt/yaml/arraylikestring.yaml diff --git a/t/fmt-yaml.t b/t/fmt-yaml.t index 85ee32ee..5c00911b 100644 --- a/t/fmt-yaml.t +++ b/t/fmt-yaml.t @@ -61,6 +61,10 @@ push @tests, 'potfile' => 'fmt/yaml/pathsoption1.pot', 'pofile' => 'fmt/yaml/pathsoption1.po', 'trans' => 'fmt/yaml/pathsoption1.trans', + }, + { + 'format' => "yaml", + 'input' => "fmt/yaml/arraylikestring.yaml", }; run_all_tests(@tests); diff --git a/t/fmt/yaml/arraylikestring.norm b/t/fmt/yaml/arraylikestring.norm new file mode 100644 index 00000000..d44041c8 --- /dev/null +++ b/t/fmt/yaml/arraylikestring.norm @@ -0,0 +1,2 @@ +--- +button: '[ copy ]' diff --git a/t/fmt/yaml/arraylikestring.po b/t/fmt/yaml/arraylikestring.po new file mode 100644 index 00000000..468690c2 --- /dev/null +++ b/t/fmt/yaml/arraylikestring.po @@ -0,0 +1,23 @@ +# SOME DESCRIPTIVE TITLE +# Copyright (C) YEAR Free Software Foundation, Inc. +# This file is distributed under the same license as the PACKAGE package. +# FIRST AUTHOR , YEAR. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: PACKAGE VERSION\n" +"POT-Creation-Date: 2023-08-24 20:47+0900\n" +"PO-Revision-Date: 2023-08-24 20:47+0900\n" +"Last-Translator: FULL NAME \n" +"Language-Team: LANGUAGE \n" +"Language: \n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. type: Hash Value: button +#: arraylikestring.yaml:1 +#, no-wrap +msgid "[ copy ]" +msgstr "[ COPY ]" diff --git a/t/fmt/yaml/arraylikestring.pot b/t/fmt/yaml/arraylikestring.pot new file mode 100644 index 00000000..c1049727 --- /dev/null +++ b/t/fmt/yaml/arraylikestring.pot @@ -0,0 +1,23 @@ +# SOME DESCRIPTIVE TITLE +# Copyright (C) YEAR Free Software Foundation, Inc. +# This file is distributed under the same license as the PACKAGE package. +# FIRST AUTHOR , YEAR. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: PACKAGE VERSION\n" +"POT-Creation-Date: 2023-08-24 20:47+0900\n" +"PO-Revision-Date: 2023-08-24 20:47+0900\n" +"Last-Translator: FULL NAME \n" +"Language-Team: LANGUAGE \n" +"Language: \n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. type: Hash Value: button +#: arraylikestring.yaml:1 +#, no-wrap +msgid "[ copy ]" +msgstr "" diff --git a/t/fmt/yaml/arraylikestring.trans b/t/fmt/yaml/arraylikestring.trans new file mode 100644 index 00000000..6d7d7dea --- /dev/null +++ b/t/fmt/yaml/arraylikestring.trans @@ -0,0 +1,2 @@ +--- +button: '[ COPY ]' diff --git a/t/fmt/yaml/arraylikestring.yaml b/t/fmt/yaml/arraylikestring.yaml new file mode 100644 index 00000000..4f9d5f3a --- /dev/null +++ b/t/fmt/yaml/arraylikestring.yaml @@ -0,0 +1 @@ +button: "[ copy ]" base-commit: c9f5cf97ebc6915d7d9c4a90707a98c8cb6ad7a2 -- 2.41.0 ```

And the test results are the following:

```shell-session $ ./Build test --test_files=t/fmt-yaml.t --verbose Deleting blib/lib/Locale/Po4a/TransTractor.pm Copying lib/Locale/Po4a/TransTractor.pm -> blib/lib/Locale/Po4a/TransTractor.pm t/fmt-yaml.t .. # Subtest: Format fmt/yaml/basic.yaml # Change directory to fmt/yaml ok 1 - Normalizing Format fmt/yaml/basic.yaml # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/basic.pot --localized t/tmp/fmt/yaml/basic.norm basic.yaml > t/tmp/fmt/yaml/basic.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating Format fmt/yaml/basic.yaml ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml --master basic.yaml --po BUILDPATH/t/fmt/yaml/basic.po --localized BUILDPATH/t/tmp/fmt/yaml/basic.trans > BUILDPATH/t/tmp/fmt/yaml/basic.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml --master basic.yaml --po BUILDPATH/t/tmp/fmt/yaml/basic.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/basic.norm.stderr BUILDPATH/t/tmp/fmt/yaml/basic.norm.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/basic.pot tmp/fmt/yaml/basic.pot > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/basic.norm tmp/fmt/yaml/basic.norm > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/basic.trans.stderr tmp/fmt/yaml/basic.trans.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/basic.trans tmp/fmt/yaml/basic.trans > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/basic.po tmp/fmt/yaml/basic.po_updated > tmp/fmt/yaml/basic.cmd_output 2>&1 1..10 ok 1 - Format fmt/yaml/basic.yaml # Subtest: basic -o keys=Name # Change directory to fmt/yaml ok 1 - Normalizing basic -o keys=Name # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/basic.pot --localized t/tmp/fmt/yaml/basic.norm -o keys=Name basic.yaml > t/tmp/fmt/yaml/basic.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating basic -o keys=Name ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml -o keys=Name --master basic.yaml --po BUILDPATH/t/fmt/yaml/keysoption1.po --localized BUILDPATH/t/tmp/fmt/yaml/basic.trans > BUILDPATH/t/tmp/fmt/yaml/basic.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml -o keys=Name --master basic.yaml --po BUILDPATH/t/tmp/fmt/yaml/basic.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/basic.norm.stderr BUILDPATH/t/tmp/fmt/yaml/basic.norm.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/keysoption1.pot tmp/fmt/yaml/basic.pot > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/basic.norm tmp/fmt/yaml/basic.norm > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/basic.trans.stderr tmp/fmt/yaml/basic.trans.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/keysoption1.trans tmp/fmt/yaml/basic.trans > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/keysoption1.po tmp/fmt/yaml/basic.po_updated > tmp/fmt/yaml/basic.cmd_output 2>&1 1..10 ok 2 - basic -o keys=Name # Subtest: basic -o keys='Name File' # Change directory to fmt/yaml ok 1 - Normalizing basic -o keys='Name File' # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/basic.pot --localized t/tmp/fmt/yaml/basic.norm -o 'keys=Name File' basic.yaml > t/tmp/fmt/yaml/basic.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating basic -o keys='Name File' ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml -o 'keys=Name File' --master basic.yaml --po BUILDPATH/t/fmt/yaml/keysoption2.po --localized BUILDPATH/t/tmp/fmt/yaml/basic.trans > BUILDPATH/t/tmp/fmt/yaml/basic.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml -o 'keys=Name File' --master basic.yaml --po BUILDPATH/t/tmp/fmt/yaml/basic.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/basic.norm.stderr BUILDPATH/t/tmp/fmt/yaml/basic.norm.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/keysoption2.pot tmp/fmt/yaml/basic.pot > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/basic.norm tmp/fmt/yaml/basic.norm > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/basic.trans.stderr tmp/fmt/yaml/basic.trans.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/keysoption2.trans tmp/fmt/yaml/basic.trans > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/keysoption2.po tmp/fmt/yaml/basic.po_updated > tmp/fmt/yaml/basic.cmd_output 2>&1 1..10 ok 3 - basic -o keys='Name File' # Subtest: -o skip_array # Change directory to fmt/yaml ok 1 - Normalizing -o skip_array # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/skiparray.pot --localized t/tmp/fmt/yaml/skiparray.norm -o keys=title -o skip_array skiparray.yaml > t/tmp/fmt/yaml/skiparray.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating -o skip_array ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml -o keys=title -o skip_array --master skiparray.yaml --po BUILDPATH/t/fmt/yaml/skiparray.po --localized BUILDPATH/t/tmp/fmt/yaml/skiparray.trans > BUILDPATH/t/tmp/fmt/yaml/skiparray.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml -o keys=title -o skip_array --master skiparray.yaml --po BUILDPATH/t/tmp/fmt/yaml/skiparray.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/skiparray.norm.stderr BUILDPATH/t/tmp/fmt/yaml/skiparray.norm.stderr > tmp/fmt/yaml/skiparray.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/skiparray.pot tmp/fmt/yaml/skiparray.pot > tmp/fmt/yaml/skiparray.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/skiparray.norm tmp/fmt/yaml/skiparray.norm > tmp/fmt/yaml/skiparray.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/skiparray.trans.stderr tmp/fmt/yaml/skiparray.trans.stderr > tmp/fmt/yaml/skiparray.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/skiparray.trans tmp/fmt/yaml/skiparray.trans > tmp/fmt/yaml/skiparray.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/skiparray.po tmp/fmt/yaml/skiparray.po_updated > tmp/fmt/yaml/skiparray.cmd_output 2>&1 1..10 ok 4 - -o skip_array # Subtest: Format fmt/yaml/utf8.yaml # Change directory to fmt/yaml ok 1 - Normalizing Format fmt/yaml/utf8.yaml # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/utf8.pot --localized t/tmp/fmt/yaml/utf8.norm -M UTF-8 utf8.yaml > t/tmp/fmt/yaml/utf8.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating Format fmt/yaml/utf8.yaml ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml -M UTF-8 --master utf8.yaml --po BUILDPATH/t/fmt/yaml/utf8.po --localized BUILDPATH/t/tmp/fmt/yaml/utf8.trans > BUILDPATH/t/tmp/fmt/yaml/utf8.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml -M UTF-8 --master utf8.yaml --po BUILDPATH/t/tmp/fmt/yaml/utf8.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/utf8.norm.stderr BUILDPATH/t/tmp/fmt/yaml/utf8.norm.stderr > tmp/fmt/yaml/utf8.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/utf8.pot tmp/fmt/yaml/utf8.pot > tmp/fmt/yaml/utf8.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/utf8.norm tmp/fmt/yaml/utf8.norm > tmp/fmt/yaml/utf8.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/utf8.trans.stderr tmp/fmt/yaml/utf8.trans.stderr > tmp/fmt/yaml/utf8.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/utf8.trans tmp/fmt/yaml/utf8.trans > tmp/fmt/yaml/utf8.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/utf8.po tmp/fmt/yaml/utf8.po_updated > tmp/fmt/yaml/utf8.cmd_output 2>&1 1..10 ok 5 - Format fmt/yaml/utf8.yaml # Subtest: basic -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array # Change directory to fmt/yaml ok 1 - Normalizing basic -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/basic.pot --localized t/tmp/fmt/yaml/basic.norm -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array basic.yaml > t/tmp/fmt/yaml/basic.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating basic -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array --master basic.yaml --po BUILDPATH/t/fmt/yaml/keysandpaths1.po --localized BUILDPATH/t/tmp/fmt/yaml/basic.trans > BUILDPATH/t/tmp/fmt/yaml/basic.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array --master basic.yaml --po BUILDPATH/t/tmp/fmt/yaml/basic.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/basic.norm.stderr BUILDPATH/t/tmp/fmt/yaml/basic.norm.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/keysandpaths1.pot tmp/fmt/yaml/basic.pot > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/basic.norm tmp/fmt/yaml/basic.norm > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/basic.trans.stderr tmp/fmt/yaml/basic.trans.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/keysandpaths1.trans tmp/fmt/yaml/basic.trans > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/keysandpaths1.po tmp/fmt/yaml/basic.po_updated > tmp/fmt/yaml/basic.cmd_output 2>&1 1..10 ok 6 - basic -o keys='Name' -o paths='Name,Level1 Dir,invoice,bill-to address city' -o skip_array # Subtest: basic -o paths='Name,level1 Dir ' -o skip_array # Change directory to fmt/yaml ok 1 - Normalizing basic -o paths='Name,level1 Dir ' -o skip_array # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/basic.pot --localized t/tmp/fmt/yaml/basic.norm -o paths='Name,Level1 Dir' -o skip_array basic.yaml > t/tmp/fmt/yaml/basic.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating basic -o paths='Name,level1 Dir ' -o skip_array ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml -o paths='Name,Level1 Dir' -o skip_array --master basic.yaml --po BUILDPATH/t/fmt/yaml/pathsoption1.po --localized BUILDPATH/t/tmp/fmt/yaml/basic.trans > BUILDPATH/t/tmp/fmt/yaml/basic.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml -o paths='Name,Level1 Dir' -o skip_array --master basic.yaml --po BUILDPATH/t/tmp/fmt/yaml/basic.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/basic.norm.stderr BUILDPATH/t/tmp/fmt/yaml/basic.norm.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/pathsoption1.pot tmp/fmt/yaml/basic.pot > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 7 - diff -u fmt/yaml/basic.norm tmp/fmt/yaml/basic.norm > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 8 - diff -uNwB fmt/yaml/basic.trans.stderr tmp/fmt/yaml/basic.trans.stderr > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 9 - diff -uN fmt/yaml/pathsoption1.trans tmp/fmt/yaml/basic.trans > tmp/fmt/yaml/basic.cmd_output 2>&1 ok 10 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/pathsoption1.po tmp/fmt/yaml/basic.po_updated > tmp/fmt/yaml/basic.cmd_output 2>&1 1..10 ok 7 - basic -o paths='Name,level1 Dir ' -o skip_array # Subtest: Format fmt/yaml/arraylikestring.yaml # Change directory to fmt/yaml ok 1 - Normalizing Format fmt/yaml/arraylikestring.yaml # Pass: perl po4a-normalize --no-deprecation -f yaml --quiet --pot t/tmp/fmt/yaml/arraylikestring.pot --localized t/tmp/fmt/yaml/arraylikestring.norm arraylikestring.yaml > t/tmp/fmt/yaml/arraylikestring.norm.stderr 2>&1 (retcode: 0) ok 2 - Translating Format fmt/yaml/arraylikestring.yaml ok 3 - Command: perl BUILDPATH/po4a-translate --no-deprecation -f yaml --master arraylikestring.yaml --po BUILDPATH/t/fmt/yaml/arraylikestring.po --localized BUILDPATH/t/tmp/fmt/yaml/arraylikestring.trans > BUILDPATH/t/tmp/fmt/yaml/arraylikestring.trans.stderr 2>&1 ok 4 - perl BUILDPATH/po4a-updatepo --no-deprecation -f yaml --master arraylikestring.yaml --po BUILDPATH/t/tmp/fmt/yaml/arraylikestring.po_updated > BUILDPATH/t/tmp/fmt/yaml/update.stderr 2>&1 # Change directory back to /home/gemmaro/src/po4a/t ok 5 - diff -uNwB fmt/yaml/arraylikestring.norm.stderr BUILDPATH/t/tmp/fmt/yaml/arraylikestring.norm.stderr > tmp/fmt/yaml/arraylikestring.cmd_output 2>&1 ok 6 - diff -u -I'^"Project-Id-Version:' -I'^"POT-Creation-Date:' -I'^"PO-Revision-Date:' -I'^\# [^[:blank:]]* translations for ' -I'^\# Language [^[:blank:]]* translations for ' -I'Copyright (C) 20.. Free Software Foundation, Inc.' -I'^\# This file is distributed under the same license as the' -I'^\# Automatically generated, 20...' fmt/yaml/arraylikestring.pot tmp/fmt/yaml/arraylikestring.pot > tmp/fmt/yaml/arraylikestring.cmd_output 2>&1 not ok 7 - Provided command (retcode: 256) # Failed test 'Provided command (retcode: 256)' # at t/Testhelper.pm line 140. # Expected retcode: 0 # FAILED command: diff -u fmt/yaml/arraylikestring.norm tmp/fmt/yaml/arraylikestring.norm > tmp/fmt/yaml/arraylikestring.cmd_output 2>&1 # Command output: # | --- fmt/yaml/arraylikestring.norm 2023-08-24 20:57:08.520463221 +0900 # | +++ tmp/fmt/yaml/arraylikestring.norm 2023-08-24 21:04:56.898560661 +0900 # | @@ -1,2 +1,2 @@ # | --- # | -button: '[ copy ]' # | +button: [ copy ] # (end of output) # Produced file tmp/fmt/yaml: # (end of tmp/fmt/yaml) # ------------------------------------------------------------- 1..7 # Looks like you failed 1 test of 7. not ok 8 - Format fmt/yaml/arraylikestring.yaml # Failed test 'Format fmt/yaml/arraylikestring.yaml' # at t/Testhelper.pm line 527. 1..8 # Looks like you failed 1 test of 8. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/8 subtests Test Summary Report ------------------- t/fmt-yaml.t (Wstat: 256 (exited 1) Tests: 8 Failed: 1) Failed test: 8 Non-zero exit status: 1 Files=1, Tests=8, 4 wallclock secs ( 0.05 usr 0.01 sys + 2.81 cusr 0.63 csys = 3.50 CPU) Result: FAIL Failed 1/1 test programs. 1/8 subtests failed. ```

I wonder if lib/Locale/Po4a/TransTractor.pm is the cause? The patch bellow seems to resolve this case (though there might be counter-examples):

```diff Subject: [PATCH 2/2] Always escape translation for hash value --- lib/Locale/Po4a/TransTractor.pm | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/lib/Locale/Po4a/TransTractor.pm b/lib/Locale/Po4a/TransTractor.pm index 70a835b6..99ce3560 100644 --- a/lib/Locale/Po4a/TransTractor.pm +++ b/lib/Locale/Po4a/TransTractor.pm @@ -1335,21 +1335,14 @@ sub handle_yaml { ( $is_yfm ? "Yaml Front Matter " : "" ) . "Hash Value:$ctx $name", "wrap" => 0 ); - if ( $el =~ /^\[.*\]$/ ) { # Do not quote the lists - $self->pushline( $header . " $translation\n" ); - } else { - - # add extra quotes to the parameter, as a protection to the extra chars that the translator could add - $self->pushline( $header . ' ' . format_scalar($translation) . "\n" ); - } + # add extra quotes to the parameter, as a protection to the extra chars that the translator could add + $self->pushline( $header . ' ' . format_scalar($translation) . "\n" ); } else { # Work around a bug in YAML::Tiny that quotes numbers # See https://github.com/Perl-Toolchain-Gang/YAML-Tiny#additional-perl-specific-notes if ( Scalar::Util::looks_like_number($el) ) { $self->pushline("$header $el\n"); - } elsif ( $el =~ /^\[.*\]$/ ) { # Do not quote the lists either - $self->pushline("$header $el\n"); } else { $self->pushline( $header . ' ' . YAML::Tiny::_dump_scalar( "dummy", $el ) . "\n" ); } -- 2.41.0 ```

po4a version: 0.69.

mquinson commented 6 months ago

Hello @gemmaro,

thanks for this bug report and patch. Please note that pull requests on github are much more comfortable for me than raw patches in the text. For once, I get an automatic testing of your change on all existing tests. If your patch seems reasonable and if all tests pass, I'm much more confident in integrating it right away.

To make sure that I merge your changes without a glitch, you should also add a test showcasing the bug that you are fixing, or a few tests of the new feature that you are adding + a note in the changelog explaining your change.

That being said, I unfortunately have a more deep concern with your patch. It is too wide as it quotes lists. It seems to me that the following yaml extract should be left unchanged by po4a:

button: "[ copy ]"
other: [ hello ]

The existing code removes the quote of the first line while your version quotes the second line. I'm not sure of how we could solve it. I always fear that we have to rewrite the parser ourselves and that this time we cannot keep using the parser that we are currently using. I'll try to think about it, but if you have any idea please speak up.

gemmaro commented 6 months ago

Hello @mquinson,

thanks for this bug report and patch. Please note that pull requests on github are much more comfortable for me than raw patches in the text. For once, I get an automatic testing of your change on all existing tests. If your patch seems reasonable and if all tests pass, I'm much more confident in integrating it right away.

To make sure that I merge your changes without a glitch, you should also add a test showcasing the bug that you are fixing, or a few tests of the new feature that you are adding + a note in the changelog explaining your change.

Thank you for the response. I will create pull requests (rather than pasting diffs) from now on.

That being said, I unfortunately have a more deep concern with your patch. It is too wide as it quotes lists. It seems to me that the following yaml extract should be left unchanged by po4a:

button: "[ copy ]"
other: [ hello ]

The existing code removes the quote of the first line while your version quotes the second line. I'm not sure of how we could solve it. I always fear that we have to rewrite the parser ourselves and that this time we cannot keep using the parser that we are currently using. I'll try to think about it, but if you have any idea please speak up.

I found that the documentation of YAML::Tiny, used in po4a, says that "Support for flow-style sequences is not required." so it is expected behavior that other: [ hello ] is not interpreted as an array. Due to the limitations of YAML::Tiny, both cases cannot be supported on the po4a side; the output is unified as either quoted or unquoted output. In summary,

original data parsed result with YAML::Tiny normalized result with po4a
hello: '[ world ]' hello: '[ world ]' hello: [ world ]
hello: [ world ] hello: '[ world ]' hello: [ world ]

Since po4a's documentation states that YAML files are to be parsed using YAML::Tiny, it seems to me that the preferred output is quoted one, and it is not a bug but a limitation in that case. It is possible that existing translations may be affected, but this can be handled by pre-processing on the user side. For example,

  1. parsing YAML with a more advanced library,
  2. serializing to a JSON string,
  3. parsing as JSON, and
  4. serializing to a YAML string,

then it would yield a simpler YAML file. After doing so, po4a (and YAML::Tiny) can manage the translation without changing the original structure.

I think there are two options. For the second of these, I'd like to create a pull request with additional test cases later.

  1. Use YAML libraries that support richer syntax

    There might be alternative libraries with more extensive YAML syntax support. However, they might have issues such as increased memory usage or portability concerns that require C bindings.

  2. Match po4a behavior as YAML::Tiny

    This is the way presented in the draft patch in this issue. Both flow-style arrays and array-like strings are treated as strings. For the former, the workaround described above can be applied.

mquinson commented 6 months ago

The problem is that some people rely on the fact that po4a leaves YAML lists unchanged. This is why the Locale::Po4a::Text module provides a yfm_skip_array option, to not translate the elements of the list. I can't remember directly but maybe f-droid webpages rely on this.

I guess that we have to go for the first solution, or even better: use our own YAML library (or use a modified version of YAML::Tiny that would be distributed as part of the po4a source code)

mquinson commented 6 months ago

I start to think that I'm completely mislead on this bug... I was trying to support the following syntax to specify a list of two elements:

- hashvalue: [ "list elem1", "list elem2"]

But the thing is that YAML:Tiny does not support this. It is parsed as a string, explaining why we fail detecting that it's a list (that should not be quoted) in po4a. This limitation of YAML::Tiny is documented: https://metacpan.org/pod/YAML::Tiny#YAML-TINY-SPECIFICATION :

Support for the "[" flow sequence indicator is not required, with the exception of empty arrays (detailed below).

So I guess you are right, we should quote all strings, even the ones beginning with [, even if it breaks some user code.

mquinson commented 6 months ago

I almost have a working patch, but I must cook right now. Stay tuned.

gemmaro commented 6 months ago

I appreciate your handling of this matter. Wishing you a happy new year!

plowsof commented 3 weeks ago

@mquinson running into problems with this change, and im not sure exactly where my issue is.

I'm not 100% certain where the problem is but my best guess is my issue is with kramdown in jekyll not removing/stripping valid* front matter.

original front matter, no problems here:

---
entry: "Merge Mining"
terms: ["merge-mine", "merge-mining", "merged-mining","merge-mined"]
summary: "The process of mining two or more blockchains at the same time"
---

po4a, does not like in line lists anymore so i try:

---
entry: "Merge Mining"
terms: 
  - "merge-mine"
  - "merge-mining"
  - "merged-mining"
  - "merge-mined"
summary: "The process of mining two or more blockchains at the same time"
---

my first wrinkle occured when i noticed the "---" change colour in my text editor indicating something was amiss with the front matter, but ok, ignoring that and building the site is ok, however, the front matter displays on the web page. whereas the original, "one liner" does not.

mquinson commented 3 weeks ago

@plowsof yamllint reports the following issue on your file.

  3:7       error    trailing spaces  (trailing-spaces)

Does removing the extra space after terms: solve your problem?

plowsof commented 3 weeks ago

thanks for the quick suggestion, sadly the problem remains. im leaning toward this being a jekyll / kramdown converter issue

if this is indeed a jekyll/kramdown issue i think we can close this and i create the issue there. here is a preview of the front matter now displaying: https://deploy-preview-14--effulgent-bombolone-f6cfb4.netlify.app/resources/moneropedia/merge-mining from https://github.com/plowsof/monero-site/pull/14/files

mquinson commented 3 weeks ago

Please try using a yaml linter on your file to see whether there is another issue elsewhere. You really want the "---" changing colour again in the text editor, showing that the syntax error is fixed.

plowsof commented 3 weeks ago

I have opened an issue on Jekyll with more information. Because it appears as though the yaml is correct, and values are being parsed from the list correctly, that this issue is jekylls problem. this issue can be closed if you agree, here is the issue, and much thanks for the quick responses: https://github.com/jekyll/jekyll/issues/9618

mquinson commented 3 weeks ago

Ok. I close this issue then. On need, please open another issue with all details, and a reference to the current one.