JMAConsulting / biz.jmaconsulting.mte

Mandrill Emails Extension for CiviCRM
10 stars 21 forks source link

Error: POST to [callback] failed with 500: #88

Closed magnolia61 closed 8 years ago

magnolia61 commented 9 years ago

From within the mandrill config screen I get the following: Error: POST to http://yourorg.org/civicrm/ajax/mte/callback?mandrillSecret=108d3af8a77fc502 failed with 500 Is there anything that I need to change in my htaccess to make this work? I use drupal 7.37 / civicrm 4.6.3

JoeMurray commented 9 years ago

Can you try changing yourorg.org to the site domain? Can you provide the values of your dirs and urls for extensions from Administer > System Settings > Directories / Resource URLs?

shawnholt commented 9 years ago

HI @JoeMurray - I see this too. I looked at apache long and returning a 500 but nothing in message_log. Drupal watchdog says Notice: Undefined index: event in CRM_Mte_BAO_Mandrill::processMandrillCalls() (line 84 of /home/xxxxx/public_html/drupal/sites/default/files/civicrm/extensions/biz.jmaconsulting.mte/CRM/Mte/BAO/Mandrill.php).

extension directory: /home/xxxxx/public_html/drupal/sites/default/files/civicrm/extensions/ and resource URL: https://xxxxx.com/sites/default/files/civicrm/extensions

Note that I did an upgrade from earlier version (which had webhooks working.)

Outbound is fine.

magnolia61 commented 9 years ago

Here's my settings: extension directory: /home/xyz//public_html/prod/sites/all/modules/civicrm_extensions/ resource URL: https://www.xyz.nl/sites/all/modules/civicrm_extensions/

shawnholt commented 9 years ago

@JoeMurray happy to help debug - pls tell me what to look for.

yosefromano commented 9 years ago

I have the same issue. In addition to the error, mandrill is sending 6 emails a day to notify about 1 bounced email that happened weeks ago

bhcohen commented 9 years ago

I'm seeing this as well: Wordpress 4.2.3, CiviCRM 4.5, Mandrill Transactional Emails 2.0

Mandrill is getting a 500 trying to post a message to my web server about a bad mailbox.

Even though the web hook gets a HTTP 500 status, the Contact's e-mail address is being marked as "On Hold Bounce"

If I disable the the Mandrill Transactional Emails extension, the web hook gets a 200 http status

MtnPavlas commented 9 years ago

Howdy guys, has there been any guidance on how to troubleshoot this? For a few weeks the webhook worked flawlessly and then it started to error out a whole lot (on July 10 or so) and has been erroring out on and off ever since. I get the following in the notification email:

Your Mandrill webhook at URL https://www.evergreenmtb.org/index.php?option=com_civicrm&task=civicrm/ajax/mte/callback&mandrillSecret='secret code' has failed with the following error: POST to https://www.evergreenmtb.org/index.php?option=com_civicrm&task=civicrm/ajax/mte/callback&mandrillSecret='secret code' failed with 500:

Thank you for any pointers ;- )

yosefromano commented 9 years ago

I've been having the same problem, haven't been able to find a fix.

On 8/13/15 10:58 PM, MtnPavlas wrote:

Howdy guys, has there been any guidance on how to troubleshoot this? For a few weeks the webhook worked flawlessly and then it started to error out a whole lot (on July 10 or so) and has been erroring out on and off ever since. I get the following in the notification email:

Your Mandrill webhook at URL https://www.evergreenmtb.org/index.php?option=com_civicrm&task=civicrm/ajax/mte/callback&mandrillSecret=41d32f466ce92a4c has failed with the following error: POST to https://www.evergreenmtb.org/index.php?option=com_civicrm&task=civicrm/ajax/mte/callback&mandrillSecret=41d32f466ce92a4c failed with 500:

Thank you for any pointers ;- )

— Reply to this email directly or view it on GitHub https://github.com/JMAConsulting/biz.jmaconsulting.mte/issues/88#issuecomment-130982378.

shawnholt commented 9 years ago

I had this problem. Disabled PHP IDS permission in drupal for anon users and it works. Not sure how this works in WP. I don't know how to tell IDS to ignore it so I (unfortunately) disabled it. check civicrm Config Logs to make sure. Also check other security setup (like modsec - i think I had to disable a rule at one point.) You can also see apache logs to debug ....

yosefromano commented 9 years ago

Shawnholt, Are you referring to this permission?

CiviCRM: skip IDS check IDS system is bypassed for users with this permission. Prevents false errors for admin users.

What is the ids check supposed to do? Is it safe to disable it? Yosef

On 8/14/15 8:21 AM, shawnholt wrote:

I had this problem. Disabled PHP IDS permission in drupal for anon users and it works. Not sure how this works in WP. I don't know how to tell IDS to ignore it so I (unfortunately) disabled it. check civicrm Config Logs to make sure.

— Reply to this email directly or view it on GitHub https://github.com/JMAConsulting/biz.jmaconsulting.mte/issues/88#issuecomment-131146342.

shawnholt commented 9 years ago

Did you try to disable it? Ref:https://github.com/PHPIDS/PHPIDS

MtnPavlas commented 8 years ago

even with disabled IDS check, I still get webhook failures (some successes, though, so that's what puzzling)

shawnholt commented 8 years ago

I have seen the same issue. Are you just using the test feature in mandrill or have you tested with actual bounces? I don't think the mandrill API test works properly.

MtnPavlas commented 8 years ago

Actually, when I use "send test", I get "successfully posted <#> of events to ..." I get many of these errors for real webhook posts: Error: POST to https://www.evergreenmtb.org/index.php?option=com_civicrm&task=civicrm/ajax/mte/callback&mandrillSecret= failed with 500:

shawnholt commented 8 years ago

I have been seeing this recently. note that in mandrill you can actually see the batch of events (need to run json_decode) and see if there is something "wrong". I still see failure quite often but not sure why or if important.

samuelsov commented 8 years ago

I have found at least one cause that make the webhook to fail with the error 500. The source contact id is determined by its email (in createActivity -> self::retrieveEmailContactId). If there is several contacts with the same email, it fails, even if the second contact is deleted.

We should at the very least remove the deleted contact from the check but in my case it's not robust enough as someone could create a new contact for the same organisation with the same email. Not sure what is the best way to go but we might want to have settings to manually define source contact.

As a quick fix, instead of enforcing the unicity of the contact, i return the first contact found (order by contact_id) and it works fine.

shawnholt commented 8 years ago

@samuelsov did you make a patch? It looks like there are (at least) 2 causes of this: see also #93

samuelsov commented 8 years ago

I could but as i have mentioned, it's a quick fix. A better solution may be to have some way to define a matching table between email and sender contact. I can work on a patch but i'd like to have much more feedback before spending too much time on this.

sanjayjaindc commented 8 years ago

Hi, I am seeing this on a WordPress 4.0 CiviCRM 4.5.5 site

I was seeing this error: Cannot break/continue 1 level in /srv/www/wp/public_html/wp-content/plugins/files/civicrm/extensions/biz.jmaconsulting.mte/CRM/Mte/BAO/Mandrill.php on line 349

I changed line 349 to "return" instead of "continue" as suggested by #93 - the webhook then worked (according to Mandrill) but Civi is not processing the bounces correctly and I am seeing this error: PHP Warning: implode(): Invalid arguments passed in /srv/www/wp/public_html/wp-content/plugins/civicrm/civicrm/CRM/Core/IDS.php on line 262

shawnholt commented 8 years ago

https://github.com/JMAConsulting/biz.jmaconsulting.mte/issues/93 is a HACK - it is not a fix. I turned off IDS and it does not affect anything in Drupal. I do not know how to disable IDS in Wordpress.

MtnPavlas commented 8 years ago
  1. turning off IDS does not solve the webhook failures for me (not to mention it's not the right thing to do)
  2. email is not a unique identifier (think family with 3 kids using the parent's email for all of them); need to figure out how we can use CiviCRM Contact ID in order to process the feedback from Mandrill to CiviCRM
vakeesan26 commented 8 years ago

I also managed to fix the post 500 Error by changing Mandrill.php line 349 to "return" instead of "continue", this might be a PHP syntax error thank you @samuelsov

shawnholt commented 8 years ago

Note @vakeesan26 - I thought that fixed it but it caused other problems and broke functionality so please test before assuming that . Syntax does look wrong though.... pls supply a patch.

vakeesan26 commented 8 years ago

I have tested in WordPress 3.6 & CiviCRM 4.5.8 , Drupal 7.38 & CiviCRM 4.6.4 until now its working properly I haven't seen any functionality issues can you please write what functionality has been broken ?

shawnholt commented 8 years ago

Sorry - it was several months ago. I was not able to get bounce processing / web-hook to work properly. If you look through thread at @sanjayjaindc comment - looks like he wasn't either. I vaguely recall that following the code didn't make sense but again it was back in Aug. If bounce is working I suggest you submit a patch.

shawnholt commented 8 years ago

To be honest, I'm kind of concerned that people are using it with that invalid php https://github.com/JMAConsulting/biz.jmaconsulting.mte/issues/93 - perhaps it works with certain php versions or settings, or they are not using webhooks/bounce processing.

olivierh65 commented 8 years ago

I've made some tests, and I've noticed that 500 errors occur when the webhook payload is large (more than 100k). In Mandrilla Webhooks Settings / View batch ==> Emulate Webhook Using cURL

The webhook payload is too large to fit on the common command-line. Download mandrill-webhook.txt and save it in the working directory before running the curl command.

If I split the downloaded files, and submit 2 curl request, there is no error !!!

Is there a limit size ?

vakeesan26 commented 8 years ago

98 is fixing this issue [Mandrill.php line 349 issue]

I have tested on CiviCRM 4.5.6 - WP 4.2.5 and CiviCRM 4.6.10 - Drupal 7

JoeMurray commented 8 years ago

Closing as we are addressing several issues that cause this in our upcoming release. Please create new issues if you get 500 errors with new release.