Project60 / org.project60.sepa

SEPA direct debit integration with civicrm
19 stars 46 forks source link

PSA: Duplicate Collection Issues with CiviSEPA 1.6.0 #634

Closed bjendres closed 1 year ago

bjendres commented 2 years ago

What's the problem?

With the rollout of CiviSEPA 1.6.0 we have been getting reports from a limited number of users where the same installment of a recurring SEPA mandate is being collected multiple times.

This happens when the transaction group with the first regular collection is closed. If you then refresh/rebuild the collection groups, you might generate another collection for the same mandate and the same date.

Why is this happening?

This happens when the SEPA mandate data in your system doesn't comply with the CiviSEPA status model (here). In those cases the already collected contribution is not identified because of a wrong status or payment instrument, and will therefore be collected again.

CiviSEPA used to be more lenient with deviations from the standard model, but the implementation of other payment instruments (#572) required a more strict approach.

It seems that the bad mandate data mostly comes from old CiviSEPA payment processors, and custom mandate imports.

How can we fix this?

Seeing that the source of the 'bad' mandates is not clear, and neither can it be ruled out that new 'bad' mandates would be generated in the future, I opted for a "self-healing" algorithm (#629) executed before each batching process.

This algorithm will try to correct common violations of the status model, and log all changes in a log file that will be presented to the user. This algorithm is still being tested, but if you're having issues right now, you can create a backup of your database, install the attached CiviSEPA version, and see if your problems go away. Feel free to let us know if you encounter any problems with this approach.

What happens next?

As soon as we get more test feedback for the "self-healing" algorithm we'll release CiviSEPA 1.6.1-beta1.

Can't I just wait with the upgrade?

Unfortunately, the CiviCRM core version 5.43+ removed some infrastructure that will cause crashes, e.g. when merging contacts with SEPA mandates (#616).

Current developer version with the "self-healing" algorithm: org.project60.sepa-1.6.0+ee6b6dd00c699.tar.gz

Detsieber commented 2 years ago

After some intensive testing and clarifying in different productive environments I can say: The issue of duplicate collections is happening only, when the "update recurring" button is pressed during the timeframe of "recurring grace". However, after applying the "self healing" algorithm, this won't happen anymore! This is very good, and please release this as a new release, @bjendres !

On the other hand, we should be aware, that legitimate FRST contributions still will be collected duplicate in this case - these can not be covered by the "self healing". But this is no reason against the improvement implemented by Björn.

bjendres commented 2 years ago

Self-healing (#572) released with 1.6.1, and will of course also be present in 1.7.x

I'll leave the ticket open to direct people to the right version.