pazz / alot

Terminal-based Mail User Agent
GNU General Public License v3.0
700 stars 163 forks source link

UnicodeEncodeError: 'ascii' codec can't encode character '\u200c' in position 7079: ordinal not in range(128) #1507

Open josch opened 4 years ago

josch commented 4 years ago

Hi,

this issue exists on the current master branch commit d9533f197bd26c9014419939b6b6c2c1dfb9a220 with python 3.8.3rc1 on Debian unstable with notmuch 0.28.4-1. It can be reproduced using the following gzipped and base64 encoded spam message: spam.txt

The md5sum of the file should be 25ef5adcab4127baf83d2870bc31408b

I then run alot like this:

PYTHONPATH=$PWD python3 -m alot --debug-level=debug --logfile=log search id:2E.14.14998.RT3421ZD@aq.mta2vrest.cc.prd.sparkpost

And am getting this in the log file:

DEBUG:ui:global command string: "search id:2E.14.14998.RT3421ZD@aq.mta2vrest.cc.prd.sparkpost"
DEBUG:__init__:mode:global got commandline "search id:2E.14.14998.RT3421ZD@aq.mta2vrest.cc.prd.sparkpost"
DEBUG:__init__:ARGS: ['search', 'id:2E.14.14998.RT3421ZD@aq.mta2vrest.cc.prd.sparkpost']
DEBUG:__init__:cmd parms {'sort': None, 'query': ['id:2E.14.14998.RT3421ZD@aq.mta2vrest.cc.prd.sparkpost']}
DEBUG:utils:unquoted header: |2DF Bitcoin News <nytdirect@nytimes.com>|
DEBUG:utils:Content-Transfer-Encoding: "quoted-printable"
DEBUG:utils:assuming Content-Transfer-Encoding: quoted-printable
ERROR:ui:Traceback (most recent call last):
  File "/tmp/alot/alot/ui.py", line 723, in apply_command
    cmd.apply(self)
  File "/tmp/alot/alot/commands/globals.py", line 125, in apply
    ui.buffer_open(buffers.SearchBuffer(ui, self.query,
  File "/tmp/alot/alot/ui.py", line 417, in buffer_open
    self.buffer_focus(buf)
  File "/tmp/alot/alot/ui.py", line 477, in buffer_focus
    self.update()
  File "/tmp/alot/alot/ui.py", line 668, in update
    self.mainloop.draw_screen()
  File "/usr/lib/python3/dist-packages/urwid/main_loop.py", line 586, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/decoration.py", line 226, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/container.py", line 1085, in render
    body = self.body.render((maxcol, maxrow-ftrim-htrim),
  File "/tmp/alot/alot/buffers/buffer.py", line 19, in render
    return self.body.render(size, focus)
  File "/usr/lib/python3/dist-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 470, in render
    middle, top, bottom = self.calculate_visible(
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 353, in calculate_visible
    self._set_focus_complete( (maxcol, maxrow), focus )
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 717, in _set_focus_complete
    return self._set_focus_first_selectable(
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 687, in _set_focus_first_selectable
    middle, top, bottom = self.calculate_visible(
  File "/usr/lib/python3/dist-packages/urwid/listbox.py", line 356, in calculate_visible
    focus_widget, focus_pos = self._body.get_focus()
  File "/tmp/alot/alot/walker.py", line 39, in get_focus
    return self._get_at_pos(self.focus)
  File "/tmp/alot/alot/walker.py", line 72, in _get_at_pos
    widget = self._get_next_item()
  File "/tmp/alot/alot/walker.py", line 85, in _get_next_item
    next_widget = self.containerclass(next_obj, **self.kwargs)
  File "/tmp/alot/alot/widgets/search.py", line 26, in __init__
    self.rebuild()
  File "/tmp/alot/alot/widgets/search.py", line 60, in rebuild
    width, part = build_text_part(partname, self.thread,
  File "/tmp/alot/alot/widgets/search.py", line 145, in build_text_part
    content = prepare_string(name, thread, maxw)
  File "/tmp/alot/alot/widgets/search.py", line 213, in prepare_string
    s = content(thread)
  File "/tmp/alot/alot/widgets/search.py", line 188, in prepare_content_string
    lastcontent = ' '.join(m.get_body_text() for m in msgs)
  File "/tmp/alot/alot/widgets/search.py", line 188, in <genexpr>
    lastcontent = ' '.join(m.get_body_text() for m in msgs)
  File "/tmp/alot/alot/db/message.py", line 280, in get_body_text
    return extract_body_part(self.mime_part)
  File "/tmp/alot/alot/db/utils.py", line 501, in extract_body_part
    rendered_payload = render_part(
  File "/tmp/alot/alot/db/utils.py", line 345, in render_part
    raw_payload = remove_cte(part)
  File "/tmp/alot/alot/db/utils.py", line 440, in remove_cte
    bp = quopri.decodestring(payload.encode('ascii'))
UnicodeEncodeError: 'ascii' codec can't encode character '\u200c' in position 7079: ordinal not in range(128)