HTML and Inline Image duplicates body of email

andrewrensburg commented 1 month ago

Hi,

When the mailmerge_template.txt looks like this:

TO: {{email}}
SUBJECT: {{broker}}
FROM: FooBar <foo@bar.co.za>
Content-Type: text/html
ATTACHMENT: {{attachment}}
ATTACHMENT: C:\Users\foobar\assets\latest_email_png.png

<html>
<body>

<p>Dear {{broker}},</p>

<p>Please find attached stuff.</p>

<p>Kind regards</p>
<p>The Stuff Team</p>
<img alt="second" src="C:\Users\foobar\assets\latest_email_png.png">

</body>
</html>

The body of the email is duplicated in the receiving mail. Interestingly, the sent mail within Outlook doesn't look duplicated, but it is received duplicated:

Had to blackspace most of the image for confidentiality.

When removing the inline img tag, the bug no longer occurs.

awdeorio commented 4 weeks ago

Hmm, I'm having trouble duplicating the problem. Would you be able to make some very tiny version with dummy data?

andrewrensburg commented 4 weeks ago

Sure thing!

Attached a mailmerge_database.csv with dummy data, the inline picture for testing, and a requirements.txt file if needed.

Requirements: [requirements.txt](https://github.com/user-attachments/files/17395

Picture: whatsapp 106/requirements.txt)

Database CSV: mailmerge_database.csv test.pdf

Mailmerge_template: mailmerge_template.txt

You'll notice on the csv, I include both my active Outlook email and Gmail account for sending. The one received on the 365 Outlook looks fine, no duplication, but gmail mail received duplicates the body of the mail.

Thanks for looking into this!

awdeorio commented 3 weeks ago

I tried to send a message using your template and database. Note that I modified some file paths. It looks like there's a problem with two email addresses separated by a semicolon.

mailmerge
...
TO: andrew@ambledown.co.za;andyjvr97@gmail.com
...

My SMTP server returned an error:

failed to send message: {'': (501, b'Syntax violates RFC 5321 section 4.1.1.3:\n"RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>" / Forward-Path ) [ SP Rcpt-parameters ] CRLF\nForward-path = Path\nPath = "<" [ A-d-l ":" ] Mailbox ">"')}

When the TO field contained just one email address, it worked for me. I wonder if that could be related to the problem? If your goal is for the message to end up in your own email archive, a trick is to include yourself as a BCC. If the goal is to send the same message to two people, a comma works (Don't forget to use quotes if this is in the CSV).

andrewrensburg commented 2 weeks ago

Hi,

Very odd, both a semicolon and a comma works for me when sending to multiple addresses.

I've just tried both using the same database (one row of semicolon and another with comma), and both rows send (in effect, I receive two emails to the first address and two to the second address).

In production, where this 'bug' was identified, including myself as the first row in the CSV are for sanity checking purposes, whereas the rest of the rows are those correctly allocated, where 99% of the time there are multiple addresses separated by semicolon.

After this current test with a comma instead of semicolon, the body is still duplicated. Removing the inline image (removing the inline image attachment makes no difference) results in a single body.

Your tests have still not resulted in duplicated email body at all?

awdeorio commented 2 weeks ago

Your tests have still not resulted in duplicated email body at all?

Right.

Next idea: send one message where you expect to see a duplicate body. use mailmerge --output-format=raw and post the output. This will contain all the details of the message and attachment, so be sure to use dummy data!

andrewrensburg commented 2 weeks ago

Thanks.

Output attached. mailmerge-raw-output.txt

awdeorio commented 2 hours ago

I'm still have trouble replicating the problem. Here's what I've tried.

Inspect the raw output you provided. It seems OK.

Compare the raw output you provided to a sample message I constructed. They seem to be comparable.


TO: awdeorio@umich.edu
FRO: awdeorio@umich.edu
SUBJECT: Broker 1
Content-Type: text/html
ATTACHENT: test.pdf
ATTACHENT: whatsapp.jpg
Content-Type: text/html

Dear Broker 1,

Please find attached relevant documentation regarding stuff.

Kind regards

The Stuff Team


3. Sent my simple message, works and looks OK.

## Next steps
My next idea is to reduce the example to the smallest possible example where the problem still occurs and post it along with the `mailmerge --output-format=raw` output.

Can you remove some or all of the attachments (you mentioned this earlier)?  One recipient?  Inline image vs. attached?

Avoid using the database and just hardcode everything in the template.  Use a dummy database that looks like this.

email _

awdeorio / mailmerge

HTML and Inline Image duplicates body of email #160