escaping issues.

[silviucpp]

Hello,

First I'm not sure this is a bug in the library. But I will appreciated help from somebody on this issues I'm facing. I have a proplists in Original and I'm expecting the encoding to produce Expected binary. But I have no clue what to do to generate this output.

When I encode the proplists the \ char from the unicode code are double encoded and everything gets broken.

-module(test).

-export([t/0]).

t() ->
    Original = [
        {<<"s1">>, <<"\\u001b test 1 2 3">>}
    ],

    Expected = <<"{\"s1\":\"\\u001b test 1 2 3\"}">>,
    Encoded = encode(Original),

    io:format("original:~p~n", [Original]),
    io:format("encoded:~p~n", [Encoded]),
    io:format("decoded:~p~n", [decode(Encoded)]),
    io:format("expected:~p~n", [decode(Expected)]).

decode(V) ->
    jsone:decode(V, [{object_format, proplist}, {allow_ctrl_chars, true}]).

encode(V) ->
    jsone:encode(V, [{float_format, [{decimals, 4}, compact]}]).

Output:

original:[{<<"s1">>,<<"\\u001b test 1 2 3">>}]
encoded:<<"{\"s1\":\"\\\\u001b test 1 2 3\"}">>
decoded:[{<<"s1">>,<<"\\u001b test 1 2 3">>}]
expected:[{<<"s1">>,<<"\e test 1 2 3">>}]

Kind regards, Silviu

[silviucpp]

As a note php jsone_encode it's working fine:

<?php
$return = json_encode(
    ["s1" => "\\u001b test 1 2 3"], 
    JSON_PRETTY_PRINT);
echo $return;

{
    "s1": "\\u001b test 1 2 3"
}

[silviucpp]

Also JS encoder works fine:

const obj = {s1: "\\u001b test 1 2 3"};
JSON.stringify(obj);

{"s1":"\\u001b test 1 2 3"}

[sile]

Hi,

It's expected behavior as the character \ in JSON strings should be escaped as described in the JSON specification. You're giving Original containing an already escaped UTF-8 string, but it's needed to pass as a non-escaped one instead (to prevent double escaping) as follows:

> Original = [{<<"s1">>, <<"\e test 1 2 3">>}].
> jsone:encode(Original).
<<"{\"s1\":\"\\u001b test 1 2 3\"}">>

> jsone:decode(jsone:encode(Original)).
#{<<"s1">> => <<"\e test 1 2 3">>}

BTW, I tried running the JS version on a Node.js shell. The result seemed to match the jsone's result.

$ node --version
v17.2.0

$ node
> const obj = {s1: "\\u001b test 1 2 3"};
> JSON.stringify(obj);
'{"s1":"\\\\u001b test 1 2 3"}'

[silviucpp]

Thanks for clarifications !

Silviu