Closed meskio closed 4 years ago
I have the same issue. It doesn't need to be encrypted though, signed messages trigger this as well. Quite a lot of time is spent loading the email's from disk, and in gpg (getting the key and verifying signatures). None of this seems necessary, and is a regression from previous versions. I might have a look later on.
These are the top calls:
ncalls tottime percall cumtime percall filename:lineno(function)
472/1 0.003 0.000 40.620 40.620 {built-in method builtins.exec}
1 0.000 0.000 40.620 40.620 <string>:1(<module>)
1 0.000 0.000 40.620 40.620 /usr/lib/python3.8/runpy.py:197(run_module)
1 0.000 0.000 40.619 40.619 /usr/lib/python3.8/runpy.py:64(_run_code)
1 0.000 0.000 40.619 40.619 /home/kevin/vcs/once/alot/alot/__main__.py:4(<module>)
1 0.000 0.000 40.188 40.188 /home/kevin/vcs/once/alot/alot/__main__.py:86(main)
1 0.000 0.000 37.591 37.591 /home/kevin/vcs/once/alot/alot/ui.py:47(__init__)
1 0.000 0.000 37.590 37.590 /usr/lib/python3/dist-packages/urwid/main_loop.py:276(run)
1 0.000 0.000 37.590 37.590 /usr/lib/python3/dist-packages/urwid/main_loop.py:374(_run)
1 0.000 0.000 37.589 37.589 /usr/lib/python3/dist-packages/urwid/main_loop.py:1328(run)
1 0.000 0.000 37.589 37.589 /usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py:265(run)
1 0.000 0.000 37.588 37.588 /usr/lib/python3.8/asyncio/base_events.py:557(run_forever)
35 0.001 0.000 37.588 1.074 /usr/lib/python3.8/asyncio/base_events.py:1784(_run_once)
44 0.000 0.000 26.248 0.597 /usr/lib/python3.8/asyncio/events.py:79(_run)
44 0.000 0.000 26.248 0.597 {method 'run' of 'Context' objects}
15 0.000 0.000 26.141 1.743 /home/kevin/vcs/once/alot/alot/ui.py:244(apply_commandline)
17/15 0.000 0.000 26.133 1.742 /home/kevin/vcs/once/alot/alot/ui.py:704(apply_command)
16 0.000 0.000 25.903 1.619 /usr/lib/python3/dist-packages/urwid/main_loop.py:574(draw_screen)
6 0.000 0.000 25.890 4.315 /home/kevin/vcs/once/alot/alot/ui.py:646(update)
6458/16 0.021 0.000 25.813 1.613 /usr/lib/python3/dist-packages/urwid/widget.py:138(cached_render)
2716/11 0.007 0.000 25.813 2.347 /usr/lib/python3/dist-packages/urwid/decoration.py:219(render)
11 0.000 0.000 25.787 2.344 /usr/lib/python3/dist-packages/urwid/container.py:1062(render)
11 0.000 0.000 25.782 2.344 /home/kevin/vcs/once/alot/alot/buffers/buffer.py:18(render)
10 0.001 0.000 25.779 2.578 /usr/lib/python3/dist-packages/urwid/listbox.py:462(render)
326 0.002 0.000 25.461 0.078 /home/kevin/vcs/once/alot/alot/walker.py:78(_get_next_item)
23/19 0.006 0.000 25.330 1.333 /usr/lib/python3/dist-packages/urwid/listbox.py:327(calculate_visible)
326 0.001 0.000 25.277 0.078 /home/kevin/vcs/once/alot/alot/widgets/search.py:20(__init__)
326 0.008 0.000 25.273 0.078 /home/kevin/vcs/once/alot/alot/widgets/search.py:31(rebuild)
1568 0.002 0.000 25.263 0.016 /home/kevin/vcs/once/alot/alot/walker.py:61(_get_at_pos)
1439 0.001 0.000 25.253 0.018 /home/kevin/vcs/once/alot/alot/walker.py:45(get_next)
1304 0.009 0.000 24.766 0.019 /home/kevin/vcs/once/alot/alot/widgets/search.py:116(build_text_part)
1304 0.005 0.000 24.688 0.019 /home/kevin/vcs/once/alot/alot/widgets/search.py:193(prepare_string)
326 0.001 0.000 24.665 0.076 /home/kevin/vcs/once/alot/alot/widgets/search.py:177(prepare_authors_string)
326 0.002 0.000 24.664 0.076 /home/kevin/vcs/once/alot/alot/db/thread.py:177(get_authors_string)
326 0.006 0.000 24.648 0.076 /home/kevin/vcs/once/alot/alot/db/thread.py:148(get_authors)
326 0.004 0.000 24.555 0.075 /home/kevin/vcs/once/alot/alot/db/thread.py:224(get_messages)
991/436 0.015 0.000 24.326 0.056 /home/kevin/vcs/once/alot/alot/db/thread.py:236(accumulate)
991 0.027 0.000 24.268 0.024 /home/kevin/vcs/once/alot/alot/db/message.py:29(__init__)
991 0.023 0.000 23.353 0.024 /home/kevin/vcs/once/alot/alot/db/message.py:99(get_email)
991 0.007 0.000 23.277 0.023 /home/kevin/vcs/once/alot/alot/db/utils.py:302(decrypted_message_from_bytes)
14 0.000 0.000 21.436 1.531 /home/kevin/vcs/once/alot/alot/ui.py:193(_apply_fire)
4 0.000 0.000 21.381 5.345 /home/kevin/vcs/once/alot/alot/commands/globals.py:170(apply)
5 0.000 0.000 20.719 4.144 /usr/lib/python3/dist-packages/urwid/listbox.py:710(_set_focus_complete)
991 0.011 0.000 16.882 0.017 /home/kevin/vcs/once/alot/alot/db/utils.py:251(_decrypted_message_from_message)
230 0.047 0.000 16.110 0.070 /home/kevin/vcs/once/alot/alot/db/utils.py:98(_handle_signatures)
35 0.001 0.000 11.338 0.324 /usr/lib/python3.8/selectors.py:451(select)
35 11.337 0.324 11.337 0.324 {method 'poll' of 'select.epoll' objects}
230 0.016 0.000 10.054 0.044 /home/kevin/vcs/once/alot/alot/db/utils.py:38(add_signature_headers)
226 0.008 0.000 9.964 0.044 /home/kevin/vcs/once/alot/alot/crypto.py:29(get_key)
226 0.012 0.000 8.585 0.038 /usr/lib/python3/dist-packages/gpg/core.py:1201(get_key)
226 0.006 0.000 8.554 0.038 /usr/lib/python3/dist-packages/gpg/gpgme.py:1336(gpgme_get_key)
226 8.548 0.038 8.548 0.038 {built-in method gpg._gpgme.gpgme_get_key}
1598 0.022 0.000 6.913 0.004 /usr/lib/python3/dist-packages/gpg/core.py:137(_funcwrap)
991 0.009 0.000 6.389 0.006 /usr/lib/python3.8/email/__init__.py:40(message_from_bytes)
991 0.002 0.000 6.377 0.006 /usr/lib/python3.8/email/parser.py:114(parsebytes)
991 0.096 0.000 6.325 0.006 /usr/lib/python3.8/email/parser.py:59(parsestr)
991 0.013 0.000 6.229 0.006 /usr/lib/python3.8/email/parser.py:41(parse)
8301 0.010 0.000 5.830 0.001 /usr/lib/python3.8/email/feedparser.py:173(feed)
230 0.009 0.000 5.739 0.025 /home/kevin/vcs/once/alot/alot/crypto.py:190(verify_detached)
9292 0.010 0.000 5.550 0.001 /usr/lib/python3.8/email/feedparser.py:178(_call_parse)
21914/9292 0.444 0.000 5.540 0.001 /usr/lib/python3.8/email/feedparser.py:218(_parsegen)
20280 0.066 0.000 4.765 0.000 /usr/lib/python3.8/email/message.py:462(get)
1 0.000 0.000 4.704 4.704 /home/kevin/vcs/once/alot/alot/commands/globals.py:110(apply)
1 0.000 0.000 4.703 4.703 /home/kevin/vcs/once/alot/alot/ui.py:403(buffer_open)
1 0.000 0.000 4.703 4.703 /home/kevin/vcs/once/alot/alot/ui.py:460(buffer_focus)
18233 0.043 0.000 4.675 0.000 /usr/lib/python3.8/email/policy.py:150(header_fetch_parse)
18693 0.036 0.000 4.653 0.000 /usr/lib/python3.8/email/headerregistry.py:592(__call__)
230 0.011 0.000 4.344 0.019 /usr/lib/python3/dist-packages/gpg/core.py:501(verify)
15865 0.038 0.000 4.243 0.000 /usr/lib/python3.8/email/message.py:564(get_content_type)
230 0.008 0.000 4.242 0.018 /usr/lib/python3/dist-packages/gpg/gpgme.py:1794(gpgme_op_verify)
230 4.234 0.018 4.234 0.018 {built-in method gpg._gpgme.gpgme_op_verify}
18693 0.083 0.000 4.153 0.000 /usr/lib/python3.8/email/headerregistry.py:195(__new__)
17812 0.105 0.000 3.820 0.000 /usr/lib/python3.8/email/headerregistry.py:444(parse)
456 0.011 0.000 2.758 0.006 /usr/lib/python3/dist-packages/gpg/core.py:185(__init__)
6384 0.019 0.000 2.730 0.000 /usr/lib/python3/dist-packages/gpg/core.py:164(__setattr__)
456 0.005 0.000 2.670 0.006 /usr/lib/python3/dist-packages/gpg/core.py:1131(home_dir)
456 0.006 0.000 2.664 0.006 /usr/lib/python3/dist-packages/gpg/core.py:1349(set_engine_info)
456 0.015 0.000 2.646 0.006 /usr/lib/python3/dist-packages/gpg/gpgme.py:1069(gpgme_ctx_set_engine_info)
456 2.631 0.006 2.631 0.006 {built-in method gpg._gpgme.gpgme_ctx_set_engine_info}
1 0.000 0.000 2.525 2.525 /home/kevin/.config/alot/hooks.py:56(exit)
1 0.000 0.000 2.525 2.525 /usr/lib/python3.8/subprocess.py:332(call)
6 0.000 0.000 2.520 0.420 /usr/lib/python3.8/subprocess.py:1074(wait)
6 0.000 0.000 2.520 0.420 /usr/lib/python3.8/subprocess.py:1772(_wait)
4 0.000 0.000 2.520 0.630 /usr/lib/python3.8/subprocess.py:1759(_try_wait)
4 2.520 0.630 2.520 0.630 {built-in method posix.waitpid}
8195 0.037 0.000 2.106 0.000 /usr/lib/python3.8/email/message.py:588(get_content_maintype)
17572 0.154 0.000 1.832 0.000 /usr/lib/python3.8/email/_header_value_parser.py:2612(parse_content_type_header)
1008842 0.274 0.000 1.204 0.000 /usr/lib/python3.8/email/feedparser.py:128(__next__)
17427 0.044 0.000 1.154 0.000 /usr/lib/python3.8/email/_header_value_parser.py:2544(parse_mime_parameters)
27655 0.104 0.000 1.088 0.000 /usr/lib/python3.8/email/_header_value_parser.py:2402(get_parameter)
A bisect resulted in 036f30ed7c993a461f1131a358e428461a8d4396
I guess this is the issue:
diff --git a/alot/db/message.py b/alot/db/message.py
index 02c059f6..9f5fafb5 100644
--- a/alot/db/message.py
+++ b/alot/db/message.py
@@ -10,7 +10,7 @@ from datetime import datetime
from notmuch import NullPointerError
from . import utils
-from .utils import extract_body
+from .utils import get_body_part, extract_body_part
from .utils import decode_header
from .attachment import Attachment
from .. import helper
@@ -68,6 +68,8 @@ class Message:
else:
self._from = '"Unknown" <>'
+ self.mime_part = get_body_part(self.get_email())
+
def __str__(self):
"""prettyprint the message"""
aname, aaddress = self.get_author()
@@ -263,8 +265,7 @@ class Message:
def get_body_text(self):
""" returns bodystring extracted from this mail """
- # TODO: allow toggle commands to decide which part is considered body
- return extract_body(self.get_email())
+ return extract_body_part(self.mime_part)
Describe the bug If thread_authors_replace_me is set every search buffer reload will need to read every single email from disk. This makes the search very slow mostly in the case of a lot of encrypted emails.
Software Versions
To Reproduce
thread_authors_replace_me
to trueI'm personally happy to keep
thread_authors_replace_me
to false in my setup, if I'm the only one affected by it we can close the issue.