Open cfcs opened 9 years ago
also invalid utf8 encoded messages should be rejected (see src/xmpp_callbacks:36 and https://github.com/diml/zed/blob/master/src/zed_utf8.mli#L37 for a possible solution)
Seems quite messy, it seems to me that it should be the task of the underlying xml library to check for encoding validity of all the data you get so that once you have passed that boundary you are sure that all the data is trusted to be UTF-8 valid.
@dbuenzli
Ah ok, makes sense didn't think about the b64 bits.
and I'm actually not entirely sure whether https://github.com/hannesm/jackline/commit/63db57cee417779915f42de970b91f8281b7af70 is needed or guaranteed...
Well if that concerns the output of the xml decoder you use; I'd keep it since it seems to me that this will input invalid UTF-8: a quick look seems to indicate that it doesn't check subsequent bytes for correct range; see e.g. the case for '\240'..'\247'
, which treats all 4 bytes encoded coded points and compare to table 3-7 in this document. This is what is used by Xmlm which should be, once I get the time, be rewritten on top of the similar code in Uutf. The latter decoder has a proof of correctness by exhaustiveness and computer heat (that's the reason why the test is commented) on a representation of the specification.
thx @dbuenzli for investigation... and yes, I intend to switch over to your xmlm (whenever I find some more spare time for this)
reopening: escape characters are not handled, and to remind myself that most of the xml from the server is used unchecked (everything apart from the parts a user can set - presence and messages)
to wrap up from #42 - we should use 0xfffd (unicode replacement character) for control sequences instead of ? (to distinguish from real ? received over the wire). or maybe just error out (as done with invalid utf8 sequences). this replacement should be done at a single place and not spread over multiple places.
plan is to replace erm_xmpp at some point - and use decent xml parser and unicode libraries... I'm currently busy with other things, but have this on my plan (most likely before any public release)
I changed validate_utf8 to work on UChars instead, and made use of the 0xFFfd character for unknown characters. As mentioned in the pull request, I'd love to see this check for non-printable characters in general at some point -- perhaps after we change xml lib. Pull request: https://github.com/hannesm/jackline/pull/43
Verify that control characters (VT100 escapes, UTF8 and the like) are stripped/handled to prevent contacts from messing up your UI.