Consider specifying a testing IDL for a mock payment UI

aestes commented 5 years ago

WebKit has converted some of the manual payment-request tests into automated tests by creating a mock payment UI in our testing harness that can be scripted from the test. Here's what the IDL looks like right now:

interface MockPaymentCoordinator {
    void setShippingAddress(MockPaymentAddress shippingAddress);
    void changeShippingOption(DOMString shippingOption);
    void changePaymentMethod(ApplePayPaymentMethod paymentMethod);
    void acceptPayment();
    void cancelPayment();

    readonly attribute ApplePayLineItem total;
    readonly attribute sequence<ApplePayLineItem> lineItems;
    readonly attribute sequence<MockPaymentError> errors;
};

This is specific to Apple Pay right now, and is missing some stuff, so obviously this wouldn't be used as-is.

But maybe we could specify something like this so that (a) the WebKit-specific tests could be upstreamed to WPT, and (b) other browsers could run them.

ianbjacobs commented 5 years ago

cc @sideshowbarker

aestes commented 5 years ago

Marcos pointed me to this: https://marcoscaceres.github.io/payment-method-web-platform-test/

ianbjacobs commented 5 years ago

Hi @aestes,

Did you get the answer to your original question? Are you able to share Webkit tests in the Payment Request test suite?

Ian

rsolomakhin commented 5 years ago

I support this.

danyao commented 5 years ago

I started looking at how we might implement such a test API in Chrome. The main thing I'd like is a way for tests to inject a payment instrument (e.g. credit card or payment app). We currently have some test hooks that can directly manipulate the database in the browser process [1]. To make this available to WPT, we'd need to expose corresponding JavaScript bindings. This raises two issues:

Increase in binary size. Is it worth it for a test-only API?
Website abusing the test API to confuse user (e.g. injecting a bogus payment instrument that makes it looks like to user that their browser is hijacked), even if they cannot directly steal money from user's existing payment types.

What do other vendors think about these?

As a data point, the Chrome Web Bluetooth folks ran into similar issues [2] and decided to put the test API behind a compile flag, so it's only available for content_shell. The downside was that the tests can't be run using the regular web platform test scripts.

[1] https://cs.chromium.org/chromium/src/chrome/browser/ui/views/payments/payment_request_browsertest_base.h?l=181 [2] https://docs.google.com/document/d/1Nhv_oVDCodd1pEH_jj9k8gF4rPGb_84VYaZ9IG8M_WY/edit#heading=h.ydr456a07ivp

marcoscaceres commented 5 years ago

foolip commented 5 years ago

@danyao can you describe the minimal required API to make this work? Would it suffice to just instruct the browser to provide mock UI-free payment methods? Would there have to be a callback to the test code when the payment instrument has been used, or could tests just try to make a payment and then look at the result?

I think there are similarities with https://github.com/web-platform-tests/wpt/issues/12046, and possibly something simple like https://w3c.github.io/permissions/#automation could suffice.

This would also allow web developers to test their code, although I don't know if that's a real ask or if they'd just mock out the entire API instead.

CC @LukeZielinski

w3c / payment-request

Consider specifying a testing IDL for a mock payment UI #799