"Delivery of non-mail file" by `notmuch insert`

[alvherre]

Hello

Not long ago I switched my mail setup to use notmuch insert (via mailfilter) instead of good old procmail, intending to avoid duplicate processing during delivery. However, since then a number of emails are reported by notmuch as "non-mail", which in my read of the code ultimately comes from g_mime_parser_construct_message rejecting the message.

This is with the Debian-packaged version of gmime, which as best as I can tell corresponds to GMime 3.2.0.

One such failing message is attached: 1547741331.eml.txt This message was posted to the pgsql-hackers@lists.postgresql.org list; you can see it as https://postgr.es/m/20190114000701.y4ttcb74jpskkcfb@alap3.anarazel.de in parsed form. It was generated by mutt. At first I thought the problem might be related to #52 but now I doubt it.

Manually scanning the message does not reveal anything obviously broken. I'm not familiar with either notmuch nor gmime, but looking at ltrace notmuch insert < thefile it seems to me that partway through reading the diffstat part, g_mime_parser_construct_message gives up and g_mime_parser_eos is called, and things go south.

I can provide more information as needed, but at this point I'm at a loss on how to debug this further and would very much welcome some input.

Thanks

[jstedfast]

The problem is likely to be the From 60dafe4dd1f112c506c5c6abd7b9fe58a2517473 Mon Sep 17 00:00:00 2001 line here:

--p7cqdlg7szwpvqud
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0001-Don-t-include-heapam.h-from-others-headers.patch"

From 60dafe4dd1f112c506c5c6abd7b9fe58a2517473 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Sun, 13 Jan 2019 15:31:58 -0800
Subject: [PATCH] Don't include heapam.h from others headers.

Lines beginning with From tell the parser that a new message has begun when using the parser in mbox mode.

The test-parser test program in gmime/tests manages to parse the message perfectly fine afaict which would seem to confirm my suspicion since it does not enable mbox mode and instead treats the input stream as a single message.

[asn-d6]

Hello, I'm still suffering from this issue and I didn't fully understand how to resolve it from the above comment. I have a procmail config that passes emails to notmuch-insert like this:

:0
* ^Return-Path:.*noreply@github\.com
| notmuch insert --create-folder --folder=github +github
:0
| notmuch insert --keep --create-folder --folder=default +default

and every now and then I get this "Delivery of non-mail file" error and end up losing emails.

How can I work around this issue? What am I doing wrong? Thanks!

[jstedfast]

@asn-d6 I'm not familiar with procmail scripting, so that doesn't mean much to me.

What I would need to see is the raw message that is giving you problems.

[asn-d6]

@asn-d6 I'm not familiar with procmail scripting, so that doesn't mean much to me.

What I would need to see is the raw message that is giving you problems.

Hello there. I'm attaching an example mail that's giving me problems.

I have more examples, but this is one that comes as a public newsletter.

fail.txt

[jstedfast]

This is the problem on line 194:

From bestsellers to adaptations, these are the books that are trending amon=

That From has a special meaning in mbox format where it tells the parser that that is the start of the next message.

[asn-d6]

This is the problem on line 194:

From bestsellers to adaptations, these are the books that are trending amon=

That From has a special meaning in mbox format where it tells the parser that that is the start of the next message.

Haha ok. That makes sense. So what's the right fix here? Not use mbox format?

That seems like a severe limitation of mbox if you can't start a line with "From" in the body :/

[jstedfast]

What is writing the message into the mbox file? Normally, the program that appends messages to mbox is supposed to munge lines beginning with "From " (e.g. ">From ").

But yes, one option is to not use mbox.

[asn-d6]

What is writing the message into the mbox file? Normally, the program that appends messages to mbox is supposed to munge lines beginning with "From " (e.g. ">From ").

Hello @jstedfast and thanks for your patience here.

It's notmuch-insert who writes the message into the filesystem and I use the following command when it comes to the attached mail above: notmuch insert --keep --create-folder --folder=default +default

And right after I get: Error: delivery of non-mail file: '/home/user/default/new/1608028737.M120006P929043.slick'

AFAIK notmuch-insert is supposed to use maildir and not mbox (and IIUC the new from above is part of maildir), so I'm not sure how come the mbox parser is invoked here. What have I done?

[jstedfast]

notmuch might be telling the parser to use mbox format even though it is actually Maildir.

[asn-d6]

notmuch might be telling the parser to use mbox format even though it is actually Maildir.

Hmm could be. Is this something I can control with a CLI switch or something? Thanks!

[jstedfast]

I'm not familiar with notmuch, so I don't know.