group plan and subscription cancellation needs double-check to ensure cancelled in Stripe, PayPal

[Alys]

Last month, shanaqui cancelled a group plan. Both they and I checked at the time to see that the termination date had been set correctly in the guild's data - it had. However this month another payment for the group plan had been made - i.e., the payments had not been automatically cancelled in Stripe. I assume that at the time the plan was cancelled in Habitica, the code attempted to cancel it in Stripe but something failed (e.g., timeout, network error, Stripe error).

I assume something similar could happen with subscriptions.

The code that cancels group plans and subscriptions made with Stripe and subscriptions made with PayPal should check a return value from Stripe/PayPal to ensure it was successful. If an error was returned or if there was no return value, it should either try again (and again and again until successful) or notify admins (and check that the notification was successful).

(NB This isn't neeeded for Amazon subscriptions or group plans because the payments are triggered from inside Habitica. It's not needed for PayPal group plans because PayPal isn't used for them.)

---

With PR #12335 the Habitica codebase has been updated to support MongoDB 4.2 and transactions making it possible to fix this issue.

Transactions allow operations on multiple documents to be executed ensuring that either all of them are executed correctly or none, making it possible to fix this issue.

An example on using transactions can be found at https://mongoosejs.com/docs/transactions.html while more info at https://docs.mongodb.com/manual/core/transactions/. If you want to work on this issue and have any question please leave a comment!

[TheHollidayInn]

Can you post the guild id?

[Alys]

GID 0bd1137f-6391-47db-822a-f08717b33445, User ID bb089388-28ae-4e42-a8fa-f0c2bfb6f779 Group Plan was cancelled around 21 February (maybe a day different depending on time zone).

[Alys]

There was another example of this reported today from someone who tried to cancel a subscription six months ago but it mostly failed (in this case because there had been payments that PayPal couldn't take due to incorrect credit/debit information in the user's PayPal account). The Habitica subscription benefits were terminated when the user cancelled the subscription in Habitica, but the payments continued in PayPal. I believe that's the correct behaviour from PayPal - it can't cancel subscriptions with failed payments because then it wouldn't be able to give the merchant the missed payments later when the user updated their credit/debit details. For this specific case, I believe the best approach for our code to take would be to recognise that PayPal had not cancelled the subscription (I assume PayPal sends back a message about that), and then to not terminated the subscription benefits and to display a clear error message to the user, either explaining what's gone wrong or (if that's too complex / too many potential causes to consider) instructing them to post to the Bug guild about it.

Note that there most likely would have been other cases of similar bugs between the time this issue was logged and now; we just haven't been recording them here.

[Alys]

Another example was reported today. The user tried to cancel their subscription last year, but failed payments in PayPal prevented PayPal from processing the cancellation, and so after the user had updated their payment details in PayPal, the payments continued (but without subscription benefits since that part of the subscription had been cancelled in Habitica). I've refunded all their payments from this year (I can't refund the earlier ones since PayPal has a limit on when refunds can be processed).

[TheHollidayInn]

Group plan doesn't use PayPal. So that would be for a normal sub.

[Alys]

Yes, that's right.

[Alys]

There's an easy way to test for this bug for the case of errors being sent back from PayPal. Add a subscription to a user account with all the details you'd expect for a recurring subscription bought through PayPal but with a made up customerId. E.g.,:

        "purchased.plan": {
            "consecutive": {
                "count": 14,
                "offset": 0,
                "gemCapExtra": 25,
                "trinkets": 48
            },
            "quantity": 1,
            "extraMonths": 0,
            "gemsBought": 0,
            "mysteryItems": [],
            "planId": "basic_earned",
            "paymentMethod": "Paypal",
            "owner": "d8f8ee9a-160e-4a42-a0f5-f18aa22457af",
            "dateUpdated": "2018-07-01T23:03:40.380Z",
            "dateTerminated": null,
            "dateCreated": "2017-06-02T11:04:41.292Z",
            "customerId": "I-123456789012"
        }

Just check first by searching in PayPal that that ID hasn't since been used! Then attempt to cancel the subscription. A server error will occur because PayPal returns an error message.

Below is an example of an error that will be thrown. This occurred when I had a fake PayPal subscription in my account and I tried to create a Group Plan (i.e., the error occurred when Habitica's code tried to cancel the user subscription so that it could add a free subscription):

2018-07-14T11:22:23.870966+00:00 heroku[router]: at=info method=POST path="/api/v4/groups/create-plan?a=a&sub=group_monthly" host=habitrpg-staging.herokuapp.com request_id=3db76d58-9ef8-4230-aaca-398f3f9312c1 fwd="123.123.123.12" dyno=web.2 connect=0ms service=4093ms status=500 bytes=573 protocol=https
2018-07-14T11:22:23.867734+00:00 app[web.2]: 2018-07-14T11:22:23.866Z - error: Error: Response Status : 400
2018-07-14T11:22:23.867754+00:00 app[web.2]:     at IncomingMessage.<anonymous> (/app/node_modules/paypal-rest-sdk/lib/client.js:130:23)
2018-07-14T11:22:23.867756+00:00 app[web.2]:     at emitNone (events.js:111:20)
2018-07-14T11:22:23.867758+00:00 app[web.2]:     at IncomingMessage.emit (events.js:208:7)
2018-07-14T11:22:23.867759+00:00 app[web.2]:     at endReadableNT (_stream_readable.js:1064:12)
2018-07-14T11:22:23.867761+00:00 app[web.2]:     at _combinedTickCallback (internal/process/next_tick.js:138:11)
2018-07-14T11:22:23.867766+00:00 app[web.2]:     at process._tickDomainCallback (internal/process/next_tick.js:218:9) method=POST, originalUrl=/api/v4/groups/create-plan?a=a&sub=group_monthly, host=habitrpg-staging.herokuapp.com, connection=close, user-agent=Mozilla, accept=application/json, text/plain, */*, accept-language=en-US,en;q=0.5, accept-encoding=gzip, deflate, br, referer=https://habitrpg-staging.herokuapp.com/group-plans, x-client=habitica-web, x-api-user=d904bd62-da08-416b-a816-ba797c9ee265, x-user-timezoneoffset=-600, content-type=application/json;charset=utf-8, authorization=Basic xxxxxxxxxxxxxxxxxxxxxxx==, x-request-id=...., x-forwarded-for=123.123.123.12, x-forwarded-proto=https, x-forwarded-port=443, via=1.1 vegur, connect-time=0, x-request-start=1531567339776, total-route-time=0, content-length=806, id=tok_....., object=token, id=card_......, object=card, address_city=null, address_country=null, address_line1=null, address_line1_check=null, address_line2=null, address_state=null, address_zip=null, address_zip_check=null, brand=Visa, country=US, customer=null, cvc_check=pass, dynamic_last4=null, exp_month=12, exp_year=2019, fingerprint=......, funding=credit, last4=4242, , name=slkd@slkdfj.co, tokenization_method=null, type=Visa, client_ip=123.123.123.12, created=1531567336, email=slkd@slkdfj.co, livemode=false, type=card, used=false, type=guild, privacy=private, name=lsakdfj, challenges=false, description=lkasdjf, paymentType=Stripe, httpCode=500, isHandledError=false, name=INVALID_PROFILE_ID, message=The profile ID is invalid, information_link=https://developer.paypal.com/docs/api/payments.billing-agreements#errors, debug_id=d7c3e325a12fb, httpStatusCode=400, httpStatusCode=400

That error condition can't occur in normal use of course but it's a way to force a PayPal error to occur so that testing can be done for any code we write to catch errors from PayPal (i.e., it more or less simulates the kind of errors that do occur for real in this Issue). I'd guess that a similar test could be done for subscriptions created through Stripe by providing an incorrect ID ("customerId": "cus_12345678912345"), and I assume you could use the same trick to test for errors from cancelling Group Plans.

[paglias]

With PR #12335 the Habitica codebase has been updated to support MongoDB 4.2 and transactions making it possible to fix this issue.

Transactions allow operations on multiple documents to be executed ensuring that either all of them are executed correctly or none, making it possible to fix this issue.

An example on using transactions can be found at https://mongoosejs.com/docs/transactions.html while more info at https://docs.mongodb.com/manual/core/transactions/. If you want to work on this issue and have any question please leave a comment!