search

thephpleague / omnipay-sagepay

Sage Pay driver for the Omnipay PHP payment processing library
MIT License
55 stars 78 forks source link

PSD2/3DSecure2/Sagepay v4 support #135

Open jontjs opened 5 years ago

jontjs commented 5 years ago

Sagepay are beginning to publish information on the PSD2/3DSecure2 aspects of their V4 VPSProtocol: https://www.sagepay.co.uk/support/38/psd2-under-direct-integration .

At the time of writing they haven't linked to a new version of the full integration document at https://www.sagepay.co.uk/support/find-an-integration-document/direct-integration-documents, perhaps because v4 doesn't hit their Test environment until July 1st 2019, and their Live environment until "later in July") , but the docs are available at https://www.sagepay.co.uk/library/document/directintegrationandprotocol4guidelinespdf .

judgej commented 5 years ago

Thank you for the heads' up. We'll jump on this (feel free to help in any way). I'll also follow up the "Sagepay Pi" v1 protocol https://test.sagepay.com/documentation/#3-d-secure API too, with an Omnipay implementation here https://packagist.org/packages/academe/sagepay

judgej commented 5 years ago

Scanning the docs, the V4 protocol has quite a few changes. The docs are in the usual long-hand form that makes it difficult to find what the changes are without going through every line manually, but I guess that needs to be done. A checklist of individual changes can be put together to work through.

There is a calendar of deadlines. 3DS is mandatory from September 2019. V1 will be supported until teh end of 2020, so having the ability to request V1 only would be good if useful to help client sites move over.

BigManatee commented 5 years ago

I've been reading into this too, they mention having v1 as a fallback in case v2 fails for any reason but I'm not sure if that fallback will still exist after 2020. Also from what I've briefly looked at, on page 82/90 from their guidelines they mention that CReq will replace PAReq

judgej commented 5 years ago

I don't believe the fallback will exist after 2020 (I could be wrong though).

The fallback in the meantime is for banks that don't support v2, so I expect there will be a lot of fallbacks to v1 happening at the start, and they will reduce to zero later as more banks release their v2 support. After 2020 ALL banks must support it, so there is no reason for a fallback.

judgej commented 5 years ago

I will need to look at this in the next few weeks, but if anyone has any input or branches or code, please post here so we can coordinate effort.

jontjs commented 5 years ago

I've had a slightly-deeper look at the V4 protocol docs and, on the face of it, the changes are in one sense fairly small, but in another sense rather tricky...

New fields we need to send at various steps of the 3DS process:

I mean "fairly small" in the sense that most are just additional scalar fields. But "rather tricky" in that I'm not sure how a server-side PHP library can be expected to know whether JS is enabled (or the others that depend on it).

judgej commented 5 years ago

I would think the server would have two entry points for payments, one using JS and one that does not require JS. The browser can redirect the user to whichever endpoint is appropriate to start the payment process.

Great list - very helpful!

lukrak commented 5 years ago

Is there an estimated date when this enhancement will be developed and available ?

judgej commented 5 years ago

No estimated time yet.

Talking to someone who has checked with Sage Pay, they say if you are using Sage Pay Server, there should be no changes needed. Sage Pay Direct there will be, though that one is much less commonly used.

I fully expect Sage Pay Form to be in the same field - because all processing happens off-site, the 3DS handling is all done offsite too.

However, this all needs testing.

pbh-stu commented 5 years ago

I have just spoken to SagePay who gave me this link which relates to direct integration (which i currently use) https://www.sagepay.co.uk/support/38/psd2-under-direct-integration

However I am planning to switch to server (iframe) based integration to reduce our PCI compliance burden. From what i've read above server integration means sagepay hosted pages and no changes required my end (i hope).

Thanks

judgej commented 5 years ago

Thanks @pbh-stu If you do find any changes needed for Sage Pay Server, but sure to let us know. From my own research, Sage Pay have not updated documents or SDKs in years, so it's all a bit of a mystery what they expect people to do.

pbh-stu commented 5 years ago

@judgej

I've just installed a laravel project and tested with

"laravel/framework": "4.2.*", "league/omnipay": "3.0.2", "omnipay/sagepay": "3.0"

I can confirm that omnipay sagepay/server does not support sagepay protocol version 4 (required for 3DS 2.0 ) as it's hard coded for version 3 in the omnipay/Message/AbstractRequest.php file and even if that was not the case, version 4.0 requires 5 or so extra fields to be sent and an extra callback handler but nothing too major!

So my hope of server/iframe method just working was a naive dream! I wonder if one of the forks have addressed it yet.

judgej commented 5 years ago

Just top update, the published Sage Pay specs are still from 2014/2015. That's pretty appalling TBH, considering how close to the go-live date for 3DSv2 we are. I'm guessing there is nothing that can be done at this stage.

judgej commented 5 years ago

@pbh-stu could you point me at the Sage Pay docs that detail the V4 protocol for Sage Pay Server? All I can find are ancient docs.

pbh-stu commented 5 years ago

Here you go:

https://www.sagepay.co.uk/support/38/psd2-under-direct-integration https://www.sagepay.co.uk/library/document/directintegrationandprotocol4guidelinespdf

Ahh sorry that's direct (but gives you an idea) ! I will ring SagePay this morning and get the links for server. I'm actually working to hack the omnipay sagepay (Server) code to work with V4 today, if i get it to work i'll post the minimum required changs to make it work.

Thanks Stuart

judgej commented 5 years ago

A few people have said that Server is not affected after ringing Sage Pay themselves. Obviously confirm for yourself, but that's what I'm working to at the moment.

Thanks for the Direct doc (which still seems to be in draft - what are they playing at?) If I understand correctly, it is only table A1 that has changed?

BigManatee commented 5 years ago

Sagepay and their docs are just not good at all. I'm currently using Sagepay Direct via dwmsw/sagepay but depending what has to be changed I'm thinking of switching to omnipay.

jontjs commented 5 years ago

https://www.sagepay.co.uk/support/16/3d-secure-v2-what-do-customers-need-to-do says:

Server integration – There will be no mandatory changes to your current integration. We will undertake some proactive steps in the coming weeks to ensure you are ready.

I've seen very little in the way of "proactive steps" from Sagepay, on any matter, for a very long time!

pbh-stu commented 5 years ago

I've just been on the phone to SagePay, they said they had planned to roll out an update for forms, server and direct (test env) this morning but had to abort, they plan to try again Friday 16th Aug 2019 at 7am

I've been playing with Server v4 this evening , i can complete orders fine with no changes (other than protocol version number) but it only ever shows 3DSv1 not v2. SagePay said this should be sorted tomorrow. I'll be calling them back tomorrow if not.

Hope that helps Stuart

pbh-stu commented 5 years ago

Having just tested server integration it's clear that sagepay have updated their systems overnight.

If you enter CHALLENGE in the cardholder name (not in code but above where you enter card number) when placing a test order (with 3ds enabled) you will now see a nice error message !

I just rang Sagepay and a very grumpy Charlotte insisted this was normal and we do not have to test 3DSV2 (server) and they will take care of it. I said this was lame as I need to prove our systems work as well as show stakeholders and sales staff the process for training.

image

Things to note:

You must at present enter STATUS201DS as the cardholder name (above the card number) to get a 3DS V1 challenge else it will just complete with no challenge.

The protocol version does NOT need to be changed to 4.0

I suspect sagepay are cutting corners with SERVER as they are not going to receive the additional fields such as when the customer registered (which is in the spec). I suspect this means more customers will be challenged than with a DIRECT integration where all additional fields are passed along to sagepay.

Hope that helps others struggling with this.

Bollom line, no changes required for 3DSv2 support with SERVER but i suspect that will change once more people realise that sagepay have cut corners.

judgej commented 5 years ago

Not all heroes wear capes :-) Thank you for doing this thorough research and writing it up. It will be a bit help. And - Charlotte - cheer up, it's Friday!

judgej commented 5 years ago

Still nothing from Sage Pay but promises. Documentation is dated back to 2015 for direct, 2017 for Pi, and yet there are warnings that we will need to change the integrations, and that all has to be live and working next month. An utter shambles. Never known anything like it.

Phunky commented 5 years ago

Looks like we've got some buffer time - https://www.fca.org.uk/news/press-releases/fca-agrees-plan-phased-implementation-strong-customer-authentication

pbh-stu commented 5 years ago

SagePay how now fixed the 3DS v2 system on test (SERVER)

If you enter CHALLENGE as the card holder name in the IFRAME you'll see the below challenge, enter STATUS201DS to see the old 3DS V1 challenge.

image

3DSv2 Lives !

BigManatee commented 5 years ago

This whole thing has still left me a bit confused (hopefully I'm not the only one!). I'm currently using the direct integration (without iframe) - is anything/much changing with that or is that a better/easier option?

judgej commented 5 years ago

@BigManatee Yes it is confusing, because Sage Pay always fail to speak with a unified voice. This means you have to pick up snatches of detail from here and there, and make assumptions about how some things work just by trial and error.

--- end rant ---

Sage Pay Direct involves your site handling the card details forms and sending the users off to the 3DS bank, so this will very much affect you. You will also need to be fully PCI acreditted, which I'm sure you are, but is worth mentioning. The PCI accreditation is the main reason I would recommend Sage Pay Direct to anyone but teh very largest of organisations.

Sage Pay Server hands all the form filling and processing for you offsite. That's what I would recommend, and - for now at least - won't need any immediate changes.

BigManatee commented 5 years ago

@judgej Thanks for the recommendation, I'll definitely take that into consideration and more than likely switch to this repo and SagePay Server.

My understanding right now is from the 14th of September companies will stop taking non 3DS payments and then will stop taking 3Dsv1 (I don't take any non 3ds payments) at the end of 2020 so I shouldn't have to do anything straight away. Sagepay are always a dream to deal with.

judgej commented 5 years ago

Yes, I believe that is correct. Sage Pay are great on the phone - responsive, informative, helpful. I just don't know why we have to keep calling them. Nothing I have ever asked them on the phone is anything that could not have been publicly documented, so I end up blogging and writing up repo issues with the details I gather in the hope it will be useful to others.

Sorry. O did say rant over ;-)

mattybaer commented 4 years ago

I have a feeling some cards are starting to fail because the direct integration isn't updated yet. In particular Monzo cards keep failing for us.

Anybody have any idea what is going on with SagePay or rather Opayo? Nothing seems to be updated on their website for direct integration docs, but says that all integrations are PSD2 ready.

https://www.opayo.co.uk/developers

pbh-stu commented 4 years ago

@mattybaer how is this manifesting, can you see anything in MSP I.E red 3D Secure icons ect. I have a monzo card and we use direct so i'll do a test on my systems. If cards are starting to be dropped this is a problem as my checkout does not have any kind of 3d secure enabled at present. We have a new checkout experience ready to go live but don't want to force customers to use 3ds yet as we will lose sales.

Edit: I can confirm my test £12:84 monno payment did go through on direct with no 3DS Enabled in MSP

lukrak commented 4 years ago

Is there going to be an update for this package to handle 3DSv2 ? If so, then ? As according to Opayo 2021 Jan 1st is a deadline for Europe

Sheaffy commented 3 years ago

This deadline is approaching very fast

"European issuers are likely to start declining electronic payment transactions that have no authentication in place. The current 3D Secure implementation [3DSv1] will continue to be supported until end of 2020 (at which time 3DSv2 becomes mandatory worldwide)."

Sheaffy commented 3 years ago

So as part of work, I have got this fully working now, and it will go through our testing pipelines later tomorrow. Let me know if anyone wants me to do a PR.

(edit) Just to be clear, I have got this working for Direct

(**edit) This code will also fall back to 3dsv1 when necessary as per the documentation. Please keep in mind, due to some of our other limitations on gateways, this is for omnipay v2.0. But this can certainly be used to update the 3.0 version. Sorry for any inconvenience.

lukrak commented 3 years ago

@Sheaffy @judgej was it already merged or not yet sorry?

Sheaffy commented 3 years ago

@lukrak I can see in the master branch on abstractRequest that the version is still protected $VPSProtocol = '3.00'; and in all the relevant places it has not been updated.

aidan-ayala commented 3 years ago

Does anyone here have an understanding of where and when we should be sending COFUsage flags?

lukrak commented 3 years ago

@judgej are you planning to merge @Sheaffy's update any time this week? I assume there was already a pull request for this? We also need to update our sites to comply with 3DSv2 during this week so wasn't sure if you are planning to merge his changes any time soon? Otherwise we will have to implement this ourselves too

Sheaffy commented 3 years ago

@lukrak Haven't had the time to do a PR due to work sorry. At the same time, what I have done is for version 2.0 of omnipay, so I'm unsure it will be for you if you are on 3.0.

judgej commented 3 years ago

I came to have a look at this, and have seen the Opayo API documentation for the first time. I must say, what a mess. I haven't been able to find much of any use so far - there seem to be huge chunks missing, diagrams spread over multiple pages with little to help navigate them. Or I'm just looking in the wrong place? Are there any links to decent documentation for upgrading from Sage Pay v3 to Opayo v4 (or is that Opayo v1 aka Sage Pay v4?)

lukrak commented 3 years ago

@judgej I'm working on this to add changes as a part of my work too. I'm using the following pdf from Opayo and this blog post. I found blog post useful to better understand all those new required fields that are browser related.

In that PDF it's useful to read pages 12-22 as it mentions any new variables. Whole workflow didn't really change so it should be quite clear for you I think. At the end of this week I should have all commits that I did for this in one place, so I think I can write a quick summary here with the changes I had to do for this over the weekend.

judgej commented 3 years ago

Brilliant, thank you @lukrak

Sheaffy commented 3 years ago

@lukrak Thats funny, that's the same blog post I found too... It's very informative.

lukrak commented 3 years ago

Sorry for a delay. So I've tried to create a PR but if you would compare my fork with yours you will see that quite a lot of things are deleted which is not right. I think that's because when I was making changes, I was not on a latest version and now when I tried to upload all files that I modified, it obviously picks up all the stuff missing.

Anyways, I'm attaching a single file which is a single raw commit that I had to do to make this package work with 3DSV2. I'm pretty sure you will be able to copy/paste most of my changes and update this package yourself @judgej .

All in all, I followed that PDF I've mentioned before, pages 12-22. And then I've used javascript from that guy's blog post I've also mentioned.

Bellow I will highlight main changes

src/Message/DirectAuthorizeRequest.php New mandatory fields:

All of the following is only required if BrowserJavascriptEnabled is set to true. All values needs to be passed from customers browser so you will need hidden input fields. Once again, use that javascript example

image

src/Message/DirectCompleteAuthorizeRequest.php - Now returns cres and threeDSSessionData when using "4.00". You still need to be compatible with "3.00". Nothing special, you just need to pass different parameters than before

src/Message/Response.php - Very similar to DirectCompleteAuthorizeRequest.php, just new parameters to pass with 3DS request. Note that according to documentation, even if you are using "4.00" API, you might still receive old "3.00" style response since not everyone has migrated to 3DSv2 yet. But simple "if" statement works just fine.

That's essentially all the main changes you need in this package. There is also src/Message/AbstractRequest.php in my commit that I didn't mention but it's self explanatory.

From actual project's point of view that wants to start using this updated package, the way I did mine, I can now pass which VPSProtocol I want to use now ("3.00" or "4.00"). I also double check IP is IPv4 before I say I want to use "4.00" due to that API restriction. If you want to use "4.00" you just need to pass 7 hidden input fields that would be populated by your javascript and that's it, you are good to use 3DSv2.

I've tested both, "3.00" and "4.00" with these changes and seems to be working fine. From testing point of view, just test it the same way you would test "3.00" and you will see that if it's "4.00" that you are using, then 3DS confirmation window looks completely different. That's the only way that I found to tell if it's 3DS v1 or v2 that SagePay is giving to you.

Also it seems that SagePay is ok if you pass all those "Browser..." variables even if you are using "3.00" protocol which is cool because you can see I'm just always passing all those new variables no matter which API version you use.

(Sorry for terrible formatting, don't have too much time)

changes in single commit.txt

pbh-stu commented 3 years ago

Just to interject: in my test implementation (not using omnipay) we have set: browserJavascriptEnabled = 'false' challengeWindowSize ='medium' seems to work perfectly (for my use case) and negates the need to pass the extra fields server side.

lukrak commented 3 years ago

I've also tested with browserJavascriptEnabled = 0, and it seems like SagePay returns exact same 3DS page no matter these settings but I assume it might have impact on a live 3DS site. But yes, I've tested and it works just fine with browserJavascriptEnabled = 0 and you don't need to pass any extra fields that relates to browser that's why I think it's handy if developer can decide if he wants to bother with all this JS part :)

pbh-stu commented 3 years ago

Just a last note for anyone only concerned about the UK, the deadline is not till the 14th Sept 2021, so plenty of time.

Sheaffy commented 3 years ago

One thing to note, if you are using stored cards there's two extra fields you now need to pass, if you are using tokens. those are initiatedType COFUsage

They are documented in the direct integration pdf. This caught me off guard because they don't tell you that those fields are required or that migrating from 3.0 to 4.0 effects stored cards.

Sheaffy commented 3 years ago

I've also tested with browserJavascriptEnabled = 0, and it seems like SagePay returns exact same 3DS page no matter these settings but I assume it might have impact on a live 3DS site. But yes, I've tested and it works just fine with browserJavascriptEnabled = 0 and you don't need to pass any extra fields that relates to browser that's why I think it's handy if developer can decide if he wants to bother with all this JS part :)

If you do have browserJavascriptEnabled=0, I would imagine that would lead to more challenges being issued as they have less information to create a fingerprint with.

Sheaffy commented 3 years ago

For those implementing this, one more thing that just caught me out. Turns out on a MacBook Pro, on chrome, colorDepth comes out to 30... which Opayo will NOT accept, causing an error on the payment. they only accept

1,4,8,15,16,24,32,48

which turns out to be very strict.

nystag commented 3 years ago

I've encountered an issue I thought I should share. If you make use of REPEAT transactions (stored cards) and accept Amex, American Express hasn't yet updated their system to work with 3DSv2 and so with the v4 protocol you won't receive a SchemeTraceID when initiating a Credential On File transaction - so you can't repeat that transaction. Opayo have suggested continuing to use the v3 protocol for Amex cards until they support 3DSv2 :/