Weird characters added when text contains Japanese characters and a line break

[kinow]

This week while I was testing PyFlow & Autosubmit, deploying suites to ecFlow, I tested adding a Variable with a Japanese character. Everything worked, including filtering.

But when I entered a line break, and restarted the ecflow_ui, then my variable got modified. You can copy these characters to test かき. Then create a variable A with these two characters.

Restart the ecflow_ui, and now you should get the exact same Variable value.

Now enter a line break between the two characters.

Restart the UI again. And now your variable value must have been modified.

The variable name is filtered, and some rule is enforced. But since the values appear to have no restriction, I would expect Japanese, Chinese, special symbols, etc, all to behave correctly and be displayed/loaded correctly in the UI too.

Cheers Bruno

[marcosbento]

The issue is related to how ecFlow UI represents the new line character, when the variable value is retrieved from the server.

ecFlow considers new line characters by replacing them with \n when creating a suite definition; and, to display variable values in the UI, new line characters are represented as \n. When using a Japanese locale, \n is displayed as ¥n, because of the separator character imposed by the locale.

Notice that the job script generation uses the actual new line character (and not the \n), meaning that even if the UI shows the variable A=some\nvalue, the following script:

echo "--%A%---"

generates the following job:

echo "--some
value--"

A note should be added to clarify this in the documentation, but no changes to the software are foreseen.