automated checking of mail delivery logs

[confirmordeny]

Sometimes public bodies claim not to have received FOI requests. When requesters write to us a skilled member of them (i.e. not me because I don't know how!) writes back and says something like:

I've checked the mail delivery logs for http://www.whatdotheyknow.com/request/[...] and I can confirm that both your messages were successfully delivered to [public authority]'s mail servers.

I just wonder whether it might be possible to show on the request page a little green light or something that shows each outward message was successful delivered. Perhaps it could also show the date and time of delivery. I have no idea how difficult it is.

It is far from high priority. I feel it would add something to the trustworthiness of the record because people could think to themselves "yes they really did received this message"

[garethrees]

Once #3214 is merged, we need to design and implement styles for the available delivery statuses (sent, delivered, failed) when showing an outgoing message's mail server logs. Probably need a little collaboration with the designers to get the variables in the right places. Off the top of my head we'll want:

<%# Print a message like "This message has been delivered successfully" %>
<p class="delivery-status-explanation"><%= @delivery_status.humanize %></p>
<%# @delivery_status.humanize # => "This message has been delivered successfully" %>

<%# Use the "simple" delivery status for the styling and icons. Returns a Symbol %>
<%# Symbol to String conversion handled automatically %>
<%# @delivery_status.simple # => :delivered %>
<%# @delivery_status.simple.to_s # => 'delivered' %>
<i class="icon icon_<%= @delivery_status.simple %>"></i>

<%# Use the "simple" delivery status for display %>
<%# You must manually convert to String so that humanize can be called %>
<%# @delivery_status.simple.to_s.humanize # => 'Delivered' %>
<strong class="icon icon_<%= @delivery_status.simple %>">
  <%= @delivery_status.simple.to_s.humanize %>
</strong>

Here's an example of a diff you can apply to make an OutgoingMessage return some sort of delivery status:

diff --git a/app/models/outgoing_message.rb b/app/models/outgoing_message.rb
index 5a14ea2..32fb701 100644
--- a/app/models/outgoing_message.rb
+++ b/app/models/outgoing_message.rb
@@ -251,6 +251,12 @@ class OutgoingMessage < ActiveRecord::Base
   #
   # Returns an Array.
   def mail_server_logs
+    log_lines = <<-EOF.strip_heredoc.split("\n")
+    2015-10-30 19:24:16 [17814] 1ZsFHb-0004dK-SM <= request-123-abc987@example.net U=alaveteli P=local S=2252 id=ogm-14+537f69734b97c-1ebd@localhost T="FOI Request about stuff" from <request-123-abc987@example.net> for authority@example.com authority@example.com
+    2015-10-30 19:24:16 [17817] 1ZsFHb-0004dK-SM => authority@example.com F=<request-123-abc987@example.net> P=<request-123-abc987@example.net> R=dnslookup T=remote_smtp S=2297 H=cluster2.gsi.messagelabs.com [127.0.0.1]:25 X=TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128 CV=no DN="C=US,ST=California,L=Mountain View,O=Symantec Corporation,OU=Symantec.cloud,CN=mail221.messagelabs.com" C="250 ok 1446233056 qp 26062 server-4.tower-221.messagelabs.com!1446233056!7679409!1" QT=1s DT=0s
+    EOF
+    logs = log_lines.map { |line| MailServerLog.new(:line => line, :info_request_id => self.info_request_id) }
+    return logs
     case AlaveteliConfiguration.mta_log_type.to_sym
     when :exim
       exim_mail_server_logs

OutgoingMessage.last.delivery_status
# => #<MailServerLog::PostfixDeliveryStatus:0x007dcb2f0 @status=:sent>

If you want a specific delivery status, you can forcefully return one:

diff --git a/app/models/outgoing_message.rb b/app/models/outgoing_message.rb
index 5a14ea2..c3c6dea 100644
--- a/app/models/outgoing_message.rb
+++ b/app/models/outgoing_message.rb
@@ -262,6 +262,7 @@ class OutgoingMessage < ActiveRecord::Base
   end

   def delivery_status
+    return ::MailServerLog::EximDeliveryStatus.new(:normal_message_delivery)
     mail_server_logs.
       map { |log| log.line(:decorate => true).delivery_status }.
         compact.

You can pick any delivery status from https://github.com/mysociety/alaveteli/blob/49f38ac9dd0be41a45edff52b734964b0d02fb10/app/models/mail_server_log/exim_delivery_status.rb#L11-L26

[garethrees]

https://github.com/mlandauer/cuttlefish/blob/master/app/models/postfix_log_line.rb might be worth a look for postfix support

[garethrees]

@crowbot is looking at postfix in #3337