search

thoughtbot / griddler

Simplify receiving email in Rails (deprecated)
http://griddler.io/
MIT License
1.38k stars 199 forks source link

Error when parsing "CC" from "message-headers" with Mailgun adapter #122

Closed wingice closed 9 years ago

wingice commented 10 years ago

Mailgun adapter: receiving mail without CC field:

It seems params["message-headers"] is a string , not an array.


Started POST "/email_processor" for 127.0.0.1 at 2014-04-01 17:11:46 -1000
Processing by Griddler::EmailsController#create as HTML
  Parameters: {"recipient"=>"ttttsd.adsb@talentlists.com",
  "sender"=>"rd@cosunet.com.cn", "subject"=>"FW: xxx",
  "from"=>"\"R&D Recruit Mail\" ",
  "Received"=>"from aaaaa ([180.16.63.178])\tby 21CN-ent7-3(MEDUSA 10.27.101.7) with ESMTP id 1396407625.17611 for ttttsd.adssdfb@talenddtlists.com\t;\tWed Apr 2 11:00:43 2014",
  "X-Envelope-From"=>"", "Hmm_source_ip"=>"10.27.101.7:36370.1075920735", "Hmm_attache_num"=>"0001",
  "Hmm_source_type"=>"SMTP", "0/x-Total-Score"=>"0:", "3/x-Brightmail-Tracker"=>"AAAAAA==", "X-Filter-Score"=>"to=<9595959594854f828594836195828d868f958d8a9495944f84908e>, score=<1396407643RPt4uhjjjjjTjj2jjLjT2LHEuf5Qbbbbb9bbCbbdb9Cd> ",
  "X-Real-From"=>"rd@cosunet.com.cn", "X-Receive-Ip"=>"180.166.63.178 rd@cosaaunet.com.cn",
  "From"=>"\"R&D Recruit Mail\" ",
  "To"=>"",
  "Subject"=>"FW: aaa", "Date"=>"Wed, 2 Apr 2014 11:00:37 +0800",
  "Message-Id"=>"<001201cf4e1f$c1e8dcd0$45ba9670$@com.cn>",
  "Mime-Version"=>"1.0",
  "Content-Type"=>"multipart/mixed; boundary=\"----=_NextPart_000_0013_01CF4E62.D00C1CD0\"",
  "X-Mailer"=>"Microsoft Office Outlook 12.0",
  "Thread-Index"=>"Ac9IL7jpKzuy9FReQkqj6X93yR9k4AF7/k6Q",
  "Content-Language"=>"zh-cn",
  "X-Mailgun-Incoming"=>"Yes",
  "X-Mailgun-Sflag"=>"No",
  "X-Mailgun-Sscore"=>"0.0",
  "X-Mailgun-Spf"=>"Neutral",
  "message-headers"=>"[[\"Received\", \"by luna.mailgun.net with SMTP mgrt 8766208669373; Wed, 02 Apr 2014 03:00:55 +0000\"], [\"X-Envelope-From\", \"\"], [\"Received\", \"from corp.21cn.com (corp.forptr.21cn.com [121.14.129.51]) by mxa.mailgun.org with ESMTP id 533b7d5e.7f508c1bd960-in2; Wed, 02 Apr 2014 03:00:46 -0000 (UTC)\"], ....]"
Completed 500 Internal Server Error in 0.5ms

NoMethodError (private method `select' called for #):
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:42:in `extract_header_cc'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:36:in `ccs'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:16:in `normalize_params'
  griddler (0.6.4) lib/griddler/adapters/mailgun_adapter.rb:10:in `normalize_params'
  griddler (0.6.4) app/controllers/griddler/emails_controller.rb:12:in `normalized_params'
  griddler (0.6.4) app/controllers/griddler/emails_controller.rb:3:in `create'
  actionpack (3.2.16) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (3.2.16) lib/abstract_controller/base.rb:167:in `process_action'
  actionpack (3.2.16) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (3.2.16) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  activesupport (3.2.16) lib/active_support/callbacks.rb:403:in `_run__3226707876278453871__process_action__698180469685376029__callbacks'
  activesupport (3.2.16) lib/active_support/callbacks.rb:405:in `__run_callback'
  activesupport (3.2.16) lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
  activesupport (3.2.16) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (3.2.16) lib/abstract_controller/callbacks.rb:17:in `process_action'
wingice commented 10 years ago

I fix the the issue by adding extra checking:


 -        headers = params['message-headers'].select do |h|
 +        message_headers = params['message-headers']
 +        message_headers = JSON.parse(message_headers) if message_headers.is_a?(String)
 +
 +        headers = message_headers.select do |h|

I don't know whether it is an acceptable fix. If it is ok, I can make a pull request from my fork.

wingice commented 10 years ago

I saw a warning in the log, it may be the cause: not translating string to Json.

There are some unicodes chars(UTF-8, Chinese) in those fields.


WARNING: Could not parse (and so ignoring) '[["Received", "by luna.mailgun.net with SMTP mgrt 8775751066621; Fri, 04 Apr 2014 08:15:34 +0000"], ["X-Envelope-From", ""], ["Received", "from corp.21cn.com (corp.forptr.21cn.com [121.14.129.38]) by mxa.mailgun.org with ESMTP id 533e6a1d.477bb20-in2; Fri, 04 Apr 2014 08:15:25 -0000 (UTC)"], ["Received", "from ip?180.166.63.178? (entas7.inner-hermes.com [10.27.101.7])\tby corp.21cn.com (HERMES) with ESMTP id 69DF9484033\tfor ; Fri, 4 Apr 2014 16:15:10 +0800 (CST)"], ["Hmm_attache_num", "0001"], ["Hmm_source_ip", "wmail.10.27.101.7.933431481"], ["Hmm_source_type", "WEBMAIL"], ["Received", "from ip<180.166.63.178> ([180.166.63.178])\tby 21CN-ent7(MEDUSA 10.27.101.7) with ESMTP id 1396599307.27191 for pktrwvwoh.hellotechnologies@talentlists.com\t;\tFri Apr 4 16:15:17 2014"], ["1/x-Total-Score", "-62:"], ["X-Filter-Score", "to=<918c959398979890894f89868d8d90958684898f908d90888a86946195828d868f958d8a9495944f84908e>, score=<1396599317Vo8FwwwwwHwwPwwswHPsXzGU60VVVVV7VVBVVYV7BYDa> "], ["X-Real-From", "rd@cosunet.com.cn"], ["X-Receive-Ip", "180.166.63.178 rd@cosunet.com.cn"], ["Date", "Fri, 4 Apr 2014 16:15:07 +0800 (CST)"], ["From", "rd@cosunet.com.cn"], ["To", "\"pktrwvwoh.hellotechnologies\" "], ["Message-Id", "<172189816.6421396599310618.JavaMail.hermes@ent-web5>"], ["Subject", "Fw: (51job.com)\u7533\u8bf7\u8d35\u516c\u53f8C# .Net \u9ad8\u7ea7\u8f6f\u4ef6\u5de5\u7a0b\u5e08\uff08\u4e0a\u6d77\uff09\uff0d\u77f3\u946b"], ["Mime-Version", "1.0"], ["Content-Type", "multipart/mixed; boundary=\"----=_Part_673_1412613781.1396599307337\""], ["Hmm_webcln_ip", "10.27.10.86"], ["X-Hermes-Sendmode", "normal"], ["X-Hermes-Set", "CC2fLruzRNbroHaQFnf1nCpRJnBb8J7v"], ["X-Mailgun-Incoming", "Yes"], ["X-Mailgun-Sflag", "No"], ["X-Mailgun-Sscore", "0.0"], ["X-Mailgun-Spf", "Neutral"]]'
marine44 commented 10 years ago

I can also replicate this issue via https://mailgun.com/cp/routes using the "Send A Sample POST" test function.

I have installed tested your changes and they has resolved the problem for me - so thanks very much!!!

I say you should make a pull request.

calebhearth commented 10 years ago

Moved to https://github.com/wingrunr21/griddler-mandrill/issues/1

wingrunr21 commented 10 years ago

@calebthompson my adapter is mandrill not mailgun

calebhearth commented 10 years ago

Whoops!

bradpauly commented 10 years ago

mailgun sends the headers as JSON and this is fixed in the extracted version.

calebhearth commented 10 years ago

Great, so should this be closed as fixed for 1.0 and on master?

calebhearth commented 10 years ago

Or, I guess along with the apart to remove the adapter from Griddler.

bradpauly commented 10 years ago

Sorry I didn't get back about this sooner. I'd also like to fix this in the non-extracted version if that would be helpful. Is the best way to do that to check out the version tagged 0.6.4 and make a branch to fix in from there?

calebhearth commented 10 years ago

Yeah, and we could cut 0.6.5 for that, but I'd intended to EOL 0.6.* as soon as 1.0.0 came out, which should be in the next couple of days. If you feel strongly, we can do it and cut a release. Up to you @bradpauly

bradpauly commented 10 years ago

Hmm, it's in a branch but I don't want to create extra work if it isn't needed. @ryanharnwell @wingice do you plan to move to 1.0? Or are there any reasons you don't want to upgrade?

marine44 commented 10 years ago

I'd be happy to upgrade to 1.0 when released.

calebhearth commented 9 years ago

This should be good in the released version.