filtered mail mysteriously appearing in inbox

[aspiers]

I just did a sync, and now when I locally query notmuch for tag:inbox with exclude_tags set to the empty string rather than deleted;spam, I see a mail with both the inbox and spam tags, yet when I check via the gmail UI and IMAP, neither show that mail in the inbox.

How is this possible? Was this perhaps a race condition of lieer against gmail first momentarily placing spam in the inbox before moving it out of the inbox, and lieer happened to pull the mail just before gmail moved it out? I would find that quite surprising, but I can't think of any other explanation.

[aspiers]

Actually, it's not just spam. I have filter rules in gmail such as the following:

followed by:

The idea being that I only want a thread from one of these mailing lists to arrive in my inbox if I'm involved in it (signalled by the manually applied involved label). This has worked perfectly for years and I've never seen any of these mailing list threads appear in my gmail inbox, even momentarily. But now that I've just started using lieer, I see a mail from the git mailing list appear in my local notmuch database with both inbox and sw/git labels.

As with the spam example above, the only explanation I can think of is that there could be some kind of race condition between lieer syncing as the email arrives in gmail but before all the rules have applied. Or is there another explanation?

[gauteh]

It could be that you have notmuch new and lieer somehow set up to add the inbox tag? With the new configuration options in notmuch this is confusing to get right, and I am unsure what is the best way to do that now.

https://github.com/gauteh/lieer/issues/277 (I see that you have already seen this issue..). Maybe you have to enable (or add) some more debug in either lieer or notmuch to figure out where that inbox tag is coming from.

[aspiers]

No, I definitely resolved that problem last week. The majority of incoming non-inbox email is not appearing in the inbox. In this issue I'm only describing a very small number of weird exceptions.

[gauteh]

It could be a collision, but then it should resolve itself on the next sync unless gmail is somehow not putting the spamming operation into its history.

[gauteh]

I do sometimes see in GMail web interface that I have email appearing in the inbox, and then have it spammed later.

[aspiers]

I do sometimes see in GMail web interface that I have email appearing in the inbox, and then have it spammed later.

Interesting! I don't think I ever noticed this myself and I've been using gmail for years.

It could be a collision, but then it should resolve itself on the next sync unless gmail is somehow not putting the spamming operation into its history.

Could be. Is there any chance of making lieer's -v output (perhaps optionally) reference not just message ids but also a brief summary (e.g. sender and subject)? That would make it a lot easier to figure out exactly what's happening during each sync.

[gauteh]

If the data is available without doing additional requests its possible :) Is that something you would like to implement yourself? My opensource-time is pretty sporadic nowadays.

[aspiers]

Maybe I can find time to look into it. If any pointers of things I should know for that spring to your mind, please let me know :)

[aspiers]

OK, many apologies for the noise - turns out this was a case of PEBKAC, a.k.a. my fault! After having solved #277 via

notmuch config set --database new.tags new

last week, earlier today I got fed up with having the new tag on all local synced messages, so I did

notmuch config set --database new.tags ''

thinking that this would stop adding any tags on sync. However notmuch treats this as unsetting new.tags, so it went back to the default value of unread;inbox.

I'm now trying a value of ' ' instead, which seems to work at first glance; at least, the config appears right:

~/.local/share/pipx/venvs/lieer $ PYTHONPATH=. ./bin/python3
Python 3.11.8 (main, Feb 08 2024, 08:03:16) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import notmuch2
>>> notmuch2.Database().config.get('new.tags')
' '