Requery

[mjg]

This mini-series addresses two issues related to the way threads are requeried: the initial query is by a search term, and further queries are by thread id.

The first patch addresses what I consider a bug: If, say, you respond to an e-mail then the search view for the inbox will show the date of your reply but sort the thread by the date of the latest message matching the inbox query. This can look strange.

The second patch uses the solution from the first patch (remembering the query) to achieve a look closer to notmuch's search result list: display numbers of matched messages within a thread in a search view.

The latter one may need some way to configure. [In fact I think I submitted or discusssed this before but couldn't find it, sorry.]

[mjg]

I really don't see the problem that you're solving, sorry.

That means I failed to describe it ;)

In a search buffer, all displayed threads originally come from matching a query. In that sense only some of the messages in a thread are actual matches since not all messages necessarily match the query - a thread is "matched" as soon as one the messages in a thread matches the query.

notmuch search date:today gives me a line thread:000000000001cf1a 11 mins. ago [6/12] Patrick Totzke; [pazz/alot] ... for example. The thread with 12 messages "mathched" because at least 1 message (in fact 6) match the query.

Ultimately, that is what I want to have in the search buffer (patch 2). But even if you don't care about that display:

If you ask for the newest datetime in a thread then it makes a difference whether you query for the thread ID alone or for that AND the original query. Before patch 1, alot sorts the seach buffer by one of those dates and displays the other. This can look quite confusing. After patch 1 always the same date is used, namely the one from the newest matching message.

[pazz]

Quoting Michael J Gruber (2021-01-01 16:35:26)

I really don't see the problem that you're solving, sorry.

That means I failed to describe it ;)

In a search buffer, all displayed threads originally come from matching a query. In that sense only some of the messages in a thread are actual matches since not all messages necessarily match the query - a thread is "matched" as soon as one the messages in a thread matches the query.

notmuch search date:today gives me a line thread:000000000001cf1a 11 mins. ago [6/12] Patrick Totzke; [pazz/alot] ... for example. The thread with 12 messages "mathched" because at least 1 message (in fact 6) match the query.

Yes, so far to old.

Ultimately, that is what I want to have in the search buffer (patch 2). But even if you don't care about that display:

If you ask for the newest datetime in a thread then it makes a difference whether you query for the thread ID alone or for that AND the original query. Before patch 1, alot sorts the seach buffer by one of those dates and displays the other. This can look quite confusing. After patch 1 always the same date is used, namely the one from the newest matching message.

I think I see now. For instance if I have messages A<B<C in two threads {A,C} and {B} and search mode sorts them by newest first then I have {A,C} newer than {B} but its date stamp derives from the oldest msg C.

Can this not be solved more easily by making sure that search mode displays the "right" date for a thread? Notice that Thread has get_newest_date and get_oldest_date. So you could make sure that if search mode sorts threads to have the newer ones above then it should display the newest date for each thread instead of the oldest one..

[mjg]

I think I see now. For instance if I have messages A<B<C in two threads {A,C} and {B} and search mode sorts them by newest first then I have {A,C} newer than {B} but its date stamp derives from the oldest msg C. Can this not be solved more easily by making sure that search mode displays the "right" date for a thread? Notice that Thread has get_newest_date and get_oldest_date. So you could make sure that if search mode sorts threads to have the newer ones above then it should display the newest date for each thread instead of the oldest one..

No. It's not a matter of "new" vs. "old". It's a matter of "all messages in thread" vs. "matched messages in thread". They are different sets of messages, and so you might get different dates (whether newest or oldest).

Now, one could forget about the search completely, of course (i.e. fix the sorting and leave the display). But if I have a search buffer say for tag:inbox then I care about the newest arrivals in my inbox much more than about my newest replies to (possible ancient) messages in my inbox. So I would say the existing sorting is correct (and the display is not).

After all, it comes down to how you view the search buffer/widget and notmuch search. Since all notmuch queries really are about messages I consider the output of notmuch search as well as the display of the search buffer to be a "thread condensed view of messages", but still a list of matched messages more than it is a list of threads. Similarly, if you run notmuch count or notmuch tag on a query then both will apply to matched messages only, not all messages from threads with matched messages. That's why there is the relatively new thread:{<notmuch query>} search term.

[mjg]

I'll check what it would take to fix the sorting issue on the other end (leave the display as is but sort accordingly); achieving the count (matched/total) could be independent of that.

[mjg]

OK, so the man page for notmuch search says this:

Search for messages matching the given search terms, and display as results the threads containing the matched messages. The output consists of one line per thread, giving a thread ID, the date of the newest (or oldest, depending on the sort option) matched message in the thread, the number of matched messages and total messages in the thread, the names of all participants in the thread, and the subject of the newest (or oldest) message.

And this is exactly what db.threads() does and - thus - the sort order which the search buffer in alot shows. But as soon as threads are queried by a search for thread id all dates are computed based on that query (i.e. all messages in the thread).

So, I guess the first question is whether the search buffer in alot should correspond to the output of notmuch search closely. I would say yes, and this PR achieves that (in terms of the displayed date and the counts; sort order unchanged).

Now, it might be useful to have a search view corresponding to all messages in threads with matched messages, especially in terms of sort order: if you tag only the first message in a thread (as is usual) then notmuch and alot (with or without this PR) will sort by that first message's date even if there are much newer messages in the thread. This might not be what you want and makes it difficult to look for something visually.

The solution is thread:{querystring}, of course (again, independent of this PR). Though you can use this already (if you know where to put quotes) it might be worthwhile to have a command "break out to all messages" or "list all messages in matched threads" which simply takes the current query of a search buffer and opens a new search buffer for thread:"{querystring}". This would round things up.

[mjg]

So, I have been using this feature a lot ( ;) ), and before merging (however likely or unlikely that is), I would like to amend this series by one more feature. It touches the same question as the question whether the whole series makes sense:

Is alot search (the search buffer view) supposed to mirror notmuch search?

My answer is obviously yes, and the series is a consequence. Note that this point of view affects also the answer to the question: Which messages should a tag action in a search buffer affect? notmuch tag <+-tags> <query> affects the matched messsages (not threads). Alot's tag action works differently from notmuch before this series, but in the same way after. And this is what I would like to address with one more commit (--full-thread or such for the tag actions), because often you want to act on the "full thread in a search buffer" without opening that thread first and then using the existing --all. I would even expect the standard key bindings to use --full-thread in order to preserve the current behavior. Added bonus: this shows up in the key help as a reminder.

[pazz]

I find this new "fullthreads" search mode command a bit strange: it just seems to replace the current query string, or rather opens up a new search buffer for a slightly modified one. Perhaps it'd be more consistent to beef up the current "prompt" command so that it can also insert the current query string in buffer mode? That way you could simulate the fullthreads command by prompt search thread:%s ?

[mjg]

I think you can consider most of my PR's on hold, except for "fix-toggletags", which I guess is merged now.

Where is a good place for a more fundamental discussion? By that I mean design principles such as:

• A search buffer for QUERY displays the results of notmuch search QUERY (which are messages sorted by thread, not threads).
• The tag command on a THREAD in a search buffer for QUERY operates on the same messages as notmuch tag QUERY AND thread:THREAD.
• ...

Even before the switch to nm2, I found alot's behaviour surprising in some cases; the switch added some more. I tried to "fix" some of this, but I guess I lack a clear picture of what behaviour to aim for.

I'm not completely happy with the behaviour of --full-thread (and requery) yet, but before working on it further it would be better to clarify the overall desigb goal.

As for the fullthread command: It turns a search buffer for QUERY (i.e. a buffer for all messages matching QUERY, i.e. those matched by notmuch search QUERY) into a search buffer matching all threads matching QUERY (i.e. a buffer for all messages in threads with messages messaging query, i.e. those matched by notmuch search thread:{QUERY}). In general this gives you different message counts and different messages to operate on (if the commands operate like notmuch does).