New Behaviors in 4.7 Leading to Possible Misbehavior in iATS Master Branch as of 3/1/16

[tamarmeir]

Hi again guys,

I know you are not officially supporting 4.7 yet, but I wanted to verify a couple of concerning things that we are currently noticing in one of our client environments running 4.7.6 (questions interspersed with the scenario):

• When an installment of a recurring subscription is declined, a pending contribution is created (e.g. the third of 12 installments of a recurring contribution scheduled for the 19th of each month)
• The following day, the system does a failure retry, this time leading to a successfully settled transaction and a completed contribution is created (e.g. on the 20th)
• The pending contribution, however, is left unchanged
• Given that pending and completed contributions are counted towards the total number of installments of the subscription, we cancelled the pending contribution

1) It would be preferable to mark the pending contribution as cancelled once the next failure retry contribution is created regardless of its status, otherwise, wouldn't the pending installments eventually lead to the premature completion of the subscription? (More on this below)

• Back to our successfully completed retry that occurred on the 20th, the next scheduled contribution date is updated to the following month based on the day of the successfully settled transaction (e.g. on the 20th) - this behavior is not different from earlier versions of the extension, but

2) it would be preferable if the original schedule was left unchanged regardless of when or if the failure retry is ever successful, keeping the next scheduled date to the 19th of the following month, meaning that there would continue to be a pending contribution for each expected installment date that would get cancelled each time a failure retry is unsuccessful, e.g.:

• Installment number 3 fails, pending contribution created, the next scheduled contribution date is updated to the 19th of the following month, but the failure retry date is advanced to the following day (the 20th)
• Let's assume that the number of failure retries is limited to 3, and the attempts to process installment number 3 continue to fail, 3 more contributions would be created - if each previous contribution is updated from pending to cancelled, there would be 3 cancelled contributions and 1 pending contribution associated with installment number 3, and the failure retry date would advance to the 20th of the following month
• It is now the 19th of the following month, installment number 4 also fails, and a pending contribution is created, but in this case, the last pending contribution associated with number 3 is not updated to cancelled and is left in pending status
• As with installment number 3, the following retries of installment number 4 continue to fail, leaving us with 3 cancelled contributions and 1 pending contribution associated with installment number 4, essentially leaving us with a pending contribution for each expected installment that never settles successfully

3) The above behavior would postpone the subscription being prematurely marked as completed until the last installment generates a pending contribution if it too fails, but it would still be prematurely marked as completed since funds were never collected for the previous installments - is there a way to keep count of the number of pending contributions generated to prevent ACH subscriptions from creating more installments than expected, but still keep the status of the subscription as "in progress" if the amount of successfully completed contributions does not equal the total amount expected on the subscription?

4) It is worth noting that the failure count remained at 0 in this case - not too familiar with the impact of that value is - any documentation on what is planned?

Thanks as always!

Tamar

[adixon]

Interesting, I didn't know the api had changed so much in 4.7 as regards recurring contributions.

A lot of what you're describing is custom iATS payments flow as per https://github.com/iATSPayments/com.iatspayments.civicrm/wiki/Recurring-Contribution-Failure-Handling but ACH/EFT is an extra wrinkle because it should always get created as pending, and only gets confirmed within a day or so, during which time there should be no additional attempts to generate payments. So I'm seeing a lot of red flags in your description, and wonder if there might be other issues at play here as well.

[tamarmeir]

Hi Alan,

As always I appreciate the quick response.

The above was experienced with a recurring credit card contribution, we do not yet have any experience with recurring ACH in 4.7, but we may shortly.

The payment flow behavior described in the link you sent is indeed the experienced behavior in 4.4.X - "unexpected server error" is listed in the source of those pending transactions, but no such source was listed in this installment, so perhaps it is a question of version change (i.e. all else being equal, there should have been an indication in the source field of the contribution, but perhaps that isn't working in 4.7)?

I will need to get back to you in regards to whether the initial transaction took place on iATS, but I understand that if not, current behavior would require the transaction to be dealt with manually (as we did), and I certainly wasted some time on blah blah, but perhaps an enhancement to add to the bucket list would be to check for a pending transaction for the same customer token that may be identified as a failed attempt for the same installment in some way? The other (numbered) enhancements suggested above, I believe, are still relevant, so add those to the bucket list as well if you agree with the logic.

Have a wonderful weekend!

[tamarmeir]

Hi Allen,

Just heard back from the client - the contribution did NOT make it to iATS, which explains why the pending transaction needed to be cancelled manually, so thanks for the clarification there and sorry if I wasted any of your time with my aspirations to become a novelist.

Failed installments that DID make it to iATS look to be behaving as expected in 4.7 - there is no extraneous pending contribution - I have submitted some clarifying questions to Eileen on JIRA: https://issues.civicrm.org/jira/browse/CRM-16417

You did mention that my description raised some red flags for you, but I wonder if that was only because I did not initially clarify that I was describing a recurring CC contribution? We do not have any clients on 4.7 that are using recurring ACH via iATS, but I will keep a close eye on those once we do and let you know if anything appears to be off.

So, to update my requests for enhancements (should you agree with the logic):

1) Moot blah blah 2) it would be preferable if the original schedule was left unchanged regardless of when or if the failure retry is ever successful (e.g. original installment date is the 19th, 3rd installment fails but is successful on the 20th, the next scheduled contribution date updates to the 19th of the following interval) 3) Is there a way keep the status of an ACH subscription as "in progress" if the amount of successfully completed contributions does not equal the total amount expected on the subscription and have the verification job update the status of the subscription once the last pending installment has been settled successfully? You mention a wrinkle, if you (or Karin) will be at CiviCon in June, perhaps it would be easier to discuss the logic in person? 4) Moot blah blah

Thanks again for your time, Tamar blah blah

[KarinG]

:-) Hi Tamar - yes I'll be at CiviCON - Looking forward to connecting with you.

[KarinG]

Hey Tamar - once we have an official 4.7 release out - and working on that - would you please update this issue - and/or close this and create new ones - for each specific component? That would be great - thank you!

[adixon]

There's a 4.7-compatible pre-release now out if you're able to test these issues.

https://github.com/iATSPayments/com.iatspayments.civicrm/releases/tag/1.5.4-alpha