[ERROR] request#get_attachment_as_html (Encoding::CompatibilityError) "incompatible encoding reregexp match (ASCII-8BIT regexp with UTF-8 string)

[schlos]

Actual behavior

An Encoding::CompatibilityError occurred in request#get_attachment_as_html:

  incompatible encoding regexp match (ASCII-8BIT regexp with UTF-8 string)
  app/models/censor_rule.rb:54:in `gsub'

-------------------------------
Request:
-------------------------------

  * URL        : http://imamopravoznati.org/en/request/7039/response/8510/attach/html/3/Dopuna%20Zahtjeva%20za%20PPI.pdf.html

-------------------------------
Backtrace:
-------------------------------

  app/models/censor_rule.rb:54:in `gsub'
  app/models/censor_rule.rb:54:in `apply_to_binary'
  lib/alaveteli_text_masker.rb:133:in `block in apply_binary_masks'
  lib/alaveteli_text_masker.rb:133:in `each'
  lib/alaveteli_text_masker.rb:133:in `reduce'
  lib/alaveteli_text_masker.rb:133:in `apply_binary_masks'
  lib/alaveteli_text_masker.rb:42:in `apply_masks'
  app/models/incoming_message.rb:306:in `apply_masks'
  app/controllers/request_controller.rb:683:in `get_attachment_as_html'
  app/controllers/request_controller.rb:619:in `cache_attachments'
  app/controllers/application_controller.rb:120:in `record_memory'
  lib/strip_empty_sessions.rb:13:in `call'

Version

http://imamopravoznati.org/version.json

• "alaveteli_version": "0.30.0.0",
• "ruby_version": "1.9.3",

Steps to reproduce

Expected behavior

[schlos]

Same as upstream issue: https://github.com/mysociety/alaveteli/issues/4064

[schlos]

Still reproducible. Alaveteli 0.33.0.0

[schlos]

Fix merged into 0.38.0.0: https://github.com/mysociety/alaveteli/releases/tag/0.38.0.0

[schlos]

Reproducible on 19-Jun-2021:

An Encoding::CompatibilityError occurred in request#get_attachment_as_html:

  incompatible character encodings: UTF-8 and ASCII-8BIT
  app/models/censor_rule.rb:54:in `gsub'

-------------------------------
Request:
-------------------------------

  * URL        : http://imamopravoznati.org/request/6272/response/7569/attach/html/4/Plan%20gospodarenja%20otpadom%20opcina%20Gradiste.pdf.html

  * HTTP Method: GET
  * IP address : [censorsed]
  * Parameters : {"controller"=>"request", "action"=>"get_attachment_as_html", "id"=>"6272", "incoming_message_id"=>"7569", "part"=>"4", "file_name"=>"Plan gospodarenja otpadom opcina Gradiste.pdf.html"}
  * Timestamp  : 2021-06-19 04:17:22 +0200
  * Server : imamopravoznati.org
  * Rails root : /data/www/imamopravoznati.org/alaveteli
  * Process: 101521

[schlos]

Not reproducible anymore on latest SW:

"alaveteli_git_commit": "7618a519c26758ad1b7ef242386a1a87c860f968", "alaveteli_version": "0.39.1.3", "ruby_version": "2.5.1",

Test case: https://imamopravoznati.org/request/6272/response/7569/attach/html/4/Plan%20gospodarenja%20otpadom%20opcina%20Gradiste.pdf.html