Emails silently failing

Submission emails delivery is failing without any indication (user interface indicates everything is fine, no exception emails either). The exact cause and nature of the problem is unknown.

This is pretty bad because submission data is pipelined directly to the emails and not saved anywhere. So, people are submitting requests thinking they're being process and FT never receives them. We might consider starting to store submission data. For the time being I was able to arduously recover submission data for ~the past 3/4 year by sifting through production log data, identifying request logs for the controller action, parsing request parameters and transforming them into CSV format for FT.

I'll leave the janky time-crunch script I slapped together to accomplish that here, just in case.

#!/usr/bin/env ruby

require 'active_support/all'
require 'csv'

LINE_REGEX = /I, \[(?<time>[^ ]+) [^ ]*\]  INFO -- : \[[^\[\]]*\] (?<content>.*)/
POST_REGEX = /Processing by FormsController#submit as HTML/
PARAM_REGEX = /Parameters: (?<params>.*)/
RESPONSE_KEY_REGEX = /(prompt|field)/
PROMPT_REGEX = /prompt_(\d+)/

def parse_submission(time, params)
  submission = {}
  params = eval(params).deep_symbolize_keys
  submission[:time] = Time.parse time
  submission[:first_name] = params[:user][:first_name]
  submission[:last_name] = params[:user][:last_name]
  submission[:email] = params[:user][:email]
  submission[:reply_to] = params[:reply_to]
  params[:responses].each do |key, value|
    match = PROMPT_REGEX.match key
    next unless match

    submission[value] = params[:responses][:"field_#{match[1]}"]
  end
  submission
end

files = ([nil] + (1..26).to_a).map { |n| "production.log#{n && ".#{n}"}" }
submissions = []
files.each do |file|
  found = false
  File.open(file).each do |line|
    line_match = LINE_REGEX.match line
    next unless line_match

    time = line_match['time']
    content = line_match['content']
    if found
      params_match = PARAM_REGEX.match content
      params = params_match['params']

      submissions << parse_submission(time, params)
      found = false
    elsif POST_REGEX =~ content
      found = true
    end
  end
end

headers = submissions.map { |sub| sub.keys }.flatten.uniq
CSV.open("data.csv", "wb") do |csv|
  csv << headers
  submissions.sort_by { |submission| submission[:time] }.reverse.each do |submission|
    csv << headers.map { |header| submission[header] }
  end
end

umts / ft-forms

Emails silently failing #176