Make HTML format emails for Alaveteli

[crowbot]

• [x] Discuss how to make this work with theming
  • We can either go for super simple styles (ie: just the logo and black text) that don't require any styling customisation…
  • Or we can require themes to define their own partial with ruby variables or CSS strings that'll be included in the core templates.
• [x] Create shared partials (_top.html, _bottom.html, _settings.html etc) @garethrees
• [x] Create example email template @garethrees
• [ ] Create individual emails
  • Anything matching /app/views/*_mailer/*.text.erb (see below)
• [ ] Customise themes
  • WDTK, AskTheEU, Turkey, Italy, Czechia
  • And more to follow?
• [ ] Litmus testing

[garethrees]

list of mailer template dirs:

$ ls -1 app/views/ | grep mailer
contact_mailer
info_request_batch_mailer
outgoing_mailer
reminder_mailer
request_mailer
track_mailer
user_mailer

[garethrees]

List of all current email templates

$ find app -regex '.*app/views/.*_mailer/.*[.]text[.]erb'
app/views/contact_mailer/add_public_body.text.erb
app/views/contact_mailer/from_admin_message.text.erb
app/views/contact_mailer/to_admin_message.text.erb
app/views/contact_mailer/update_public_body_email.text.erb
app/views/contact_mailer/user_message.text.erb
app/views/info_request_batch_mailer/batch_sent.text.erb
app/views/outgoing_mailer/_followup_footer.text.erb
app/views/outgoing_mailer/followup.text.erb
app/views/outgoing_mailer/initial_request.text.erb
app/views/reminder_mailer/public_holidays.text.erb
app/views/request_mailer/comment_on_alert.text.erb
app/views/request_mailer/comment_on_alert_plural.text.erb
app/views/request_mailer/external_response.text.erb
app/views/request_mailer/fake_response.text.erb
app/views/request_mailer/new_response.text.erb
app/views/request_mailer/new_response_reminder_alert.text.erb
app/views/request_mailer/not_clarified_alert.text.erb
app/views/request_mailer/old_unclassified_updated.text.erb
app/views/request_mailer/overdue_alert.text.erb
app/views/request_mailer/requires_admin.text.erb
app/views/request_mailer/stopped_responses.text.erb
app/views/request_mailer/very_overdue_alert.text.erb
app/views/track_mailer/event_digest.text.erb
app/views/user_mailer/already_registered.text.erb
app/views/user_mailer/changeemail_already_used.text.erb
app/views/user_mailer/changeemail_confirm.text.erb
app/views/user_mailer/confirm_login.text.erb

[garethrees]

• A bit of example code in https://github.com/mysociety/alaveteli/tree/3420-html-emails. You can generate this email by signing up for a new account.
• More about ActionMailer.
• If you're using vagrant, set USE_MAILCATCHER_IN_DEVELOPMENT: true in config/general.yml. Run mailcatcher with bundle exec mailcatcher -f --http-ip 0.0.0.0. You should then be able to access it at http://10.10.10.30:1080/.
• You might need a valid recaptcha key for the RECAPTCHA_ values in config/general.yml
• I think premailer or roadie will be the best way to integrate styles, but I haven't read up on how to get that working.

[kingqueen3065]

pls may I request that the HTML is turn offable. I hate HTML email for lots of reasons.

[zarino]

@garethrees: https://github.com/mysociety/alaveteli/commit/d88da0552bee963e701fddd12af9000a471bf03b adds a template and styling for the example signup email. Looks like this in all the major email clients: https://litmus.com/pub/e008253 (logo is broken, obvs, because I haven't attached it to the email).

What’s the next most sensible thing to do?

[garethrees]

pls may I request that the HTML is turn offable. I hate HTML email for lots of reasons.

@kingqueen3065 the standard Rails thing is to send a multipart email containing both plaintext and HTML email (we'll continue to maintain both). You should be able to set some sort of "prefer plaintext" option in your email client for it to render the plain version. Does that sound okay to you?

[garethrees]

Quick way of sending a confirmation email from the rails console:

UserMailer.confirm_login(User.last, {:email=>"Then you can sign in to Alaveteli", :email_subject=>"Confirm your account on Alaveteli"}, 'http://example.com').deliver

[garethrees]

What’s the next most sensible thing to do?

I've pushed some commits that move from using helpers to the asset pipeline for styling.

I think the next thing to do is to implement another HTML template from a different mailer (TrackMailer, maybe?) to see how we need to separate out the stylesheets.

[zarino]

to see how we need to separate out the stylesheets.

I'm anticipating a single stylesheet shared by all the emails. They're all going to look largely the same, right?

[garethrees]

I'm anticipating a single stylesheet shared by all the emails.

Yeah, probably. In which case I think we probably want something like:

.
├── _custom.scss        # For theme settings overrides
├── _settings.scss      # Default settings
├── application.css     # Asset pipeline manifest
├── contact_mailer.scss # Styles specific to contact_mailer – might not need initially
├── main.scss           # All the default styles go here
└── user_mailer.scss    # Styles specific to user_mailer – might not need initially

[kingqueen3065]

thanks. yes, dual format emails are great, particularly if the plaintext version is readable. I often find that companies with less togetherness than MySoc only look at the HTML format and the plaintext is not readable. I am a dinosaur and prefer text-only everything (or most things) but I realise that I'm behind the times!

[garethrees]

Sneak peek :eyes:

[garethrees]

Event digest (super basic):

# IDs will vary
info_request = InfoRequest.find(120)
track_thing = TrackThing.find(11)

xapian_object = ActsAsXapian::Search.new([InfoRequestEvent], track_thing.track_query,
                                         :sort_by_prefix => 'described_at',
                                         :sort_by_ascending => true,
                                         :collapse_by_prefix => nil,
                                         :limit => 100)

TrackMailer.event_digest(info_request.user, [[track_thing, xapian_object.results, xapian_object]]).deliver

Event digest for a search track:

user = User.where(:email => 'joe@localhost').first

# Clear any records of the user being sent mail
UserInfoRequestSentAlert.where(:user_id => user.id).destroy_all
TrackThingsSentEmail.where(:user_id => user.id).destroy_all

# Make sure the user can be mailed
user.update_attributes!(
  :last_daily_track_email => (Time.now - 10.days),
  :receive_email_alerts => true,
  :email_bounced_at => nil,
  :email_bounce_message => ''
)

# Create a search track for any activity mentioning "council"
track_thing =
  TrackThing.where(:tracking_user_id => user.id,
                   :track_query => 'council',
                   :track_medium => 'email_daily',
                   :track_type => 'search_query').first_or_create

xapian_object = ActsAsXapian::Search.new([InfoRequestEvent], track_thing.track_query,
                                         :sort_by_prefix => 'described_at',
                                         :sort_by_ascending => true,
                                         :collapse_by_prefix => nil,
                                         :limit => 100)

TrackMailer.event_digest(user, [[track_thing, xapian_object.results, xapian_object]]).deliver

[garethrees]

New response mail. Nice and easy!

info_request = InfoRequest.find(105)
incoming_message = info_request.incoming_messages.last
RequestMailer.new_response(info_request, incoming_message).deliver

[garethrees]

Overdue alert might be a nice one too:

info_request = InfoRequest.find(111)
RequestMailer.overdue_alert(info_request, info_request.user).deliver

[garethrees]

A user support comment after https://github.com/mysociety/theyworkforyou/pull/1580:

"You've recently changed the format of your TheyWorkForYou emails, I think, and I'm finding them SO much more accessible now. It's perhaps a small thing, but I really appreciate it. Just wanted to let you know."

[mdeuk]

A user support comment after mysociety/theyworkforyou#1580:

"You've recently changed the format of your TheyWorkForYou emails, I think, and I'm finding them SO much more accessible now. It's perhaps a small thing, but I really appreciate it. Just wanted to let you know."

+1. We get a fair few users on WDTK who don't realise how our emails work, or what they are actually being prompted to do.

Being able to use formatting, colours, and other stylistic cues (sparingly) would perhaps bring our alerts into a format our users are accustomed to seeing on other services - and reduce our support mail that comes from misunderstood transactional messages.

The examples provided by GDS are a good starter for ten, and are quite flexible. These are likely to be very familiar to users in the UK, particularly given their use on the various Coronavirus service messaging.

https://github.com/alphagov/email-template

I would, of course, be keen to ensure that we send mail with a suitable plain text part - so that users who prefer to view their mail in plain text aren't disadvantaged. The TWFY alerts do this nicely. 👍

[schlos]

What remains to be done to finalize html emails? How can I help?

[garethrees]

Eek, that's a tough question to give an answer to as I haven't thought deeply about this since … 2016 🙀!

I think my approach would be to keep this simple in the extreme. Literally just the bare minimum HTML formatting with no images or styling (like the example commit (https://github.com/mysociety/alaveteli/commit/dcd97145d21e5bd2cdd29f9a69d2c7da01ebcbce) from our WIP branch).

I'd probably break down the work to ~a mailer at a time (UserMailer, RequestMailer, etc) – or even a single mailer action at a time. I don't mind if we start using multipart HTML emails for some things and keep only plain text for others.

The less we try to do in a single pull request, the more likely we avoid issues that stall getting it shipped.

Doing the basics isn't too much of a difficult job in principle (see a screencast of a basic mailer) – it's just that we have a lot of emails to add, and they're a little tricky to preview. We do have some developer previews at /rails/mailers, but it's not exhaustive.

[schlos]

Hi @garethrees , I saw that you did some changes to UserMailer, RequestMailer in WIP branch, so is that working now for sending html emails?

Did I understood correctly that now all that's needed is to build all HTML files for all email templates? If that is only thing, I can help and support you with HTML files. Although I can't help with Ruby functions and I'm lacking RoR knowledge :) And if email files are only one needed, do you expect to get PR's into that WIP branch which is a bit old, or PR's should come to develop branch?

Thanks, Miroslav

[garethrees]

Did I understood correctly that now all that's needed is to build all HTML files for all email templates? If that is only thing, I can help and support you with HTML files.

I think so, yeah! I don't think there will be too much "Ruby" knowledge needed, as that will already exist in the plain text version of each email template. I think the things to be done are:

• Create a "layout" html template for each mailer (app/views/layouts/*_mailer.html.erb
• Duplicate each plain text template to a html version – so e.g. wrap each paragraph in <p> tags, use <li> tags around lists, wrap links in <a> tags, and so on.

in WIP branch, so is that working now for sending html emails … do you expect to get PR's into that WIP branch which is a bit old, or PR's should come to develop branch?

The WIP branch is over 6 years old, so getting that branch actually working will be too difficult – I'd start a new branch from develop. That said, I imagine cherry-picking dcd97145d21e5bd2cdd29f9a69d2c7da01ebcbce on to a new branch has a high chance of working and a good starting point for UserMailer.

Maybe give it a try for one or two templates to see how you get on?

Thanks for taking a look at this – I think this would be a huge improvement! 🙏