XMLRPC call to jail is incorrectly encoded

[serasset]

Hi,

We are using a docker based jail written in python to handle XMLRPC run/eval requests. However, mod_vpl sends ill-formed XML data.

As an example, lets say that we have a file containing "été", the XML will be :

<member>
        <name>README.md</name>
        <value>
         <string>&#195;&#169;t&#195;&#169;</string>
        </value>
</member>

Which is incorrect, because the single "é" character is replaced by 2 bytes (the utf-8 form) that are encoded as if they were chars. This is a kind of double encoding (1 char = 2 bytes) --> (2 chars encoded using XML entities).

In the mod_vpl own jail, it works, because the XML is parsed using and ad-hoc program that wil double decode the texte (2 chars) --> (2 bytes, 1 char).

In some cases this works in our jail, due to python discrepancies (the XML parser uses expat which itself returns an array of bytes that are interpreted as utf-8 chars, hence there is also a kind of double decoding). But as expat will also validate the char entities it decodes, it will break when some specific chars are met, this will happen if one enter a greek character in any file.

We could also use a hand crafted parser of the XML that will double decode without checking anything (as does the original jail code) but as the original problems comes from an erroneous encoding, this won't solve the problems for others attempting to build a jail in another language.

This is somehow related to BUG #139 (which raises an analoguous problem but on JSON encoding).

[serasset]

The root of the issue comes from the xmlrpc_encode_request( $action, $data, ['encoding' => 'UTF-8'] ) php method that gives incorrect encoding for non ascii chars.

You should try xmlrpc_encode_request( $action, $data, ['encoding' => 'UTF-8', 'escaping' => 'markup'] ).

[jcrodriguez-dis]

Resolved in VPL 4.1.1