search

wangp / bower

A curses terminal client for the Notmuch email system
Other
119 stars 11 forks source link

Some emails fail parsing: null character not allowed #32

Open haasn opened 6 years ago

haasn commented 6 years ago

Error parsing notmuch response: line 1, column 1006: null character not allowed

I've gotten this once or twice over the years. I suspect there might be a single trailing null character causing us issues. It's possible we could make a best effort to strip trailing null in order to reduce the chance of his happening to practically 0.

wangp commented 6 years ago

An alternative to stripping might be to replace NUL with e.g. U+FFFD in unicode_escape_sequence (use char.det_from_int(0xfffd) if you want to try it).

Edit: actually json.unicode_escape_sequence only recognises escape sequences in JSON strings; as long as it doesn't fail, it doesn't matter what character it returns. The actual replacement happens in json.unescape_loop which replaces any invalid escape sequences (it treats NUL as invalid) with U+FFFD.