Extract quoted section removal to library and make quoted section markers translatable

[garethrees]

@petterreinholdtsen submitted https://github.com/mysociety/alaveteli/pull/2654 to add a quoted section marker in their language.

I don't see the benefit in continually adding additional strings to this method, as many of them will be irrelevant for a given install.

The particular markers are theme-specific, so we should supply an interface for them to be customised from a theme. This would have the benefit of reducing the performance cost of processing messages.

A nice interface to add markers would be something along the lines of:

Alaveteli::QuotedSectionRemoval.end_of_message_strings << '''----*\s*Opprinnelig melding\s*----*'''

Usage of the library in alaveteli core should look something like:

class IncomingMessage
  def self.remove_quoted_sections(text, replacement = "FOLDED_QUOTED_SECTION")
   MessageFilter::RemoveQuotedSection.new(text, replacement).apply
  end
end

There also looks like some special handling for other UK-specific stuff.

It would be awesome if you could build up a filter chain for sanitising your messages:

# config/initializers/my_theme_message_filtering.rb

IncomingMessage.filter_chain = [ RemoveLotusQuotes,
                                 RemoveQuotedSection,
                                 RemoveBBCFurtherCommsConsent,
                                 HideEmailAddresses ]

[garethrees]

This is a similar problem to #2551.

[garethrees]

• https://github.com/Ibsciss/ruby-middleware
• http://stackoverflow.com/questions/747913/why-would-i-ever-use-a-chain-of-responsibility-over-a-decorator
• http://codereview.stackexchange.com/questions/85587/building-a-chain-of-responsibility-in-ruby-to-apply-transformators-on-an-object
• https://gist.github.com/martindemello/7231bf0f407ca428b509
• http://www.devinterface.com/blog/en/2010/07/design-patterns-in-ruby-chain-of-responsibility/
• http://www.sitepoint.com/introduction-to-chain-of-responsibility/
• https://en.wikipedia.org/wiki/Chain-of-responsibility_pattern
• http://confreaks.tv/videos/railsconf2013-simple-and-elegant-rails-code-with-functional-style
• http://confreaks.tv/videos/mwrc2012-rack-middleware-as-a-general-purpose-abstraction
• http://www.blrice.net/blog/2015/09/18/a-middleware-stack-without-rack/

[petterreinholdtsen]

I revisited #2654 today, to see what I could to do make the message view nicer on Mimes brønn, but reading through this issue again leave me mystified about how to solve it. Is there some documentation somewhere explaining how to customize the original_message setting?

[garethrees]

Hi Petter,

You can override the full method in your theme.

# THEME_ROOT/lib/model_patches.rb

# -*- encoding : utf-8 -*-
Rails.configuration.to_prepare do
  InfoRequest.class_eval do
    def self.remove_quoted_sections(text, replacement = "FOLDED_QUOTED_SECTION")
      # method body omitted for demonstration
    end
  end
end

I tend to copy the full method from the commit you're deployed at, then you can make the change in 99aeb8ff1ed96bf437183e5513b6ac8db7927f77 against the overridden version in the theme.

Here's an example we've overridden in WDTK:

• Original "core" method: https://github.com/mysociety/alaveteli/blob/9828223/app/models/public_body.rb#L528-L531
• Theme override: https://github.com/mysociety/whatdotheyknow-theme/blob/5200c49/lib/model_patches.rb#L49-L71

[garethrees]

Or try https://github.com/mysociety/alaveteli/issues/1722