Richer negotiation re: address redaction?

[samuelweiler]

As I think about how "billing addresses" are used in practice in the US: sometimes portions of them (e.g. only a postal (ZIP) code) are used by card issuers for validation. Sometimes merchants want complete addresses for other reasons. Does it make sense to have a requester explain exactly what address elements it wants, and why?

Consolidated explanation of my concern below at: https://github.com/w3c/payment-request/issues/842#issuecomment-473907871

[ianbjacobs]

Hi @samuelweiler,

Can you say more about the use cases?

Our initial choice of redaction was driven by the use case of "tax computation." At least in version 1, that seems like a good starting point. After consideration of more use cases, we can see whether some merchant/user negotiation would add valuable.

Ian

[samuelweiler]

In the US, some merchants, particularly ones that are not shipping a physical product, request (only) a "credit card billing ZIP code", which appears to be used as part of the card acceptance validation. I'm imagining that it would be convenient - and lead to less transaction abandonment - if the merchant could ask for only the info needed (e.g. just the post code) and not anything else.

I also imagine that different regions have quite different patterns of practices like this, hence the suggestion for a more generalized negotiation mechanism.

[npdoty]

I'm not sure user-driven negotiation is particularly useful here (although generally I like that as a strategy) since the user doesn't typically know or set the rules for required data to complete a payment or shipment. But merchant-driven minimization of information would definitely fall under the principle of data minimization, and that can do things like improve user comfort and even encourage payments. It sounded like this feature had been proposed for a future version and maybe Ian could link us to the corresponding issue.

[samuelweiler]

npdoty: the problem I see is that merchants often desire more data than is, in fact, required. Making easier for merchants to get the data (by defaulting to sending it) is a lose for the human sending the payment.

[npdoty]

@samuelweiler but are end users in a good position to know what information is required, or to do anything besides refuse payment altogether if the site wants information that might not be necessary?

I think there are some times that negotiation might be a good approach even though it can be complicated to specify and implement, but I think those are cases where users have a lot of insight and there can be progressive levels of functionality based on level of information. But payment seems complex, externally regulated and mostly binary, though perhaps the analog example of haggling over prices would be a non-binary example.

[samuelweiler]

@npdoty Contemplate the analog example of going into a Radio Shack (back in the day), Staples (now), or many grocery stores. They'll ask for a phone number, but they're still willing to make a sale without one. I can imagine an online merchant preferring to have some bit of data but being willing to make do without.

[ianbjacobs]

Hi @samuelweiler and @npdoty,

I mentioned on the call today that the WG has considered the idea of an array provided by the merchant that says "I don't want response data fields x and y for payment method P." https://github.com/w3c/payment-request/issues/97

For the moment the industry demand to exclude CVC has been small, so there was little appetite to include this as a standard feature in Payment Request available to all payment methods.

One could also define this feature at the level of a particular payment method. As an example, for Basic Card [1] one could enable the merchant in the request data to say "Please don't collect the CVV." However, the demand for this has been sufficiently small that we have not yet chosen to do so.

I generally think of it this way:

• Experiment at in payment method definitions.
• When there is a clear demand for the same functionality across payment methods, elevate it to Payment Request API.

So a future version of Basic Card (or any other payment method) could try out this feature without requiring changes to Payment Request API.

Ian

[1] https://www.w3.org/TR/payment-method-basic-card/

[samuelweiler]

Acknowledging that I'm not swapped in on the payments markets elsewhere in the world, I imagine there's enough variation that this 'wait for clear demand' approach will lead to some (unhelpful, unwanted, and possibly harmful) convergence, driven by the US payment market. Or, perhaps even less desirably, to less adoption of this technology worldwide. Building in the flexibility from the start seems like a better approach.

[adrianhopebailie]

We should distinguish between data required for the payment and the data requested by the merchant as part of checkout.

E.g. Billing address is actually needed to authorize a card payment in many markets but we also provide a mechanism for the merchant to ask for an email address, phone number etc.

In this API we don't have any influence over payment specific data (either in the request or response). Billing address is used in the card payment flow but is:

• unique to certain territories
• used as a fraud indicator along with potentially many other data points that are decided by the card industry

If there is a desire to make it possible that data like email and phone number can be requested but marked as optional then we should look at changing the PaymentOptions dictionary so that the fields are not boolean but rather some enum like 'required' | 'optional' | undefined.

[ianbjacobs]

@adrianhopebailie,

I like your idea (for PR API after v1) and also would extend that functionality to payment method response data (relates to issue #97).

Ian

[ianbjacobs]

@samuelweiler, for now I am labeling this a "feature after v1" and we'll discuss it at our April FTF meeting. Thanks!

[samuelweiler]

I think this is pretty clearly related to the features being added in this revision: https://lists.w3.org/Archives/Public/public-payments-wg/2019Feb/0003

I would like to see this issue addressed at the present revision.

[ianbjacobs]

Hi @samuelweiler,

I will consider this a Formal Objection raised by you (arising through PING review) and carry it forward to the Directory.

Ian

[marcoscaceres]

@samuelweiler, @npdoty, what I've done thus far is heavily redact the billing address information sufficiently so it works with fraud detection systems, and allow for tax calculations.

Tax calculations, performed when the payment instrument changes: https://github.com/w3c/payment-method-basic-card/pull/67

Fraud prevention: https://github.com/w3c/payment-request/pull/846

I share the concerns about requestBillingAddress, but instead of an enum, as @adrianhopebailie suggests, how about we use either a boolean or an array?

So:

• requestBillingAddress is false (this is default): Merchant gets nothing.
• requestBillingAddress is true: Merchant gets redacted billingAddress sufficient for fraud prevention and tax calculation.
• requestBillingAddress is Array of members that the merchant needs (and redacted list still applies). So, "requestBillingAddress": ["recipient", "postcode"] would ignore "recipient" because it's on the redact list, and thus only gives the merchant back the post code. Gives merchant more control if they need to do tax calculations and/or fraud prevention checks.

As this is a significant change to the API, I don't think we should do this in V1. We should do this in v1.1 tho. Using the array should be backwards compatible with the 1.0, in as far as an array will be treated as true by the 1.0 API.

[marcoscaceres]

The other practical thing we could do for privacy is make the redactList mandatory. Right now, it's optional... and some UAs literally leak all the things: https://github.com/w3c/payment-request/issues/648#issuecomment-348377487

[samuelweiler]

I've presented a use case above where only part of the address is needed (modeled on what I see some sites doing now), and I've postulated that these needs will be different in different markets around the world. I'm opposed to making a change (at v1 or otherwise) that defaults to increasing the amount of information over what is sent now, with no obvious way for payer and payee to negotiate about this information exchange.

@marcoscaceres If you don't want to make changes at v1, perhaps then you omit requestBillingAddress from v1?

[ianbjacobs]

@samuelweiler, we added this functionality because there is otherwise a risk that the displayed total is incorrect, which makes the API unusable.

[marcoscaceres]

What @ianbjacobs said. It’s not practical to remove it because of tax calculations. Note that it’s also not very different from having a regular web form that is automatically filled by the browser.

I do understand the data sharing concern: but by the user taking explicit action to “pay”, there is a reasonable expectation of negotiation and understanding in any transaction that billing address may affect tax, that shipping address will affect shipping cost (along with shipping options), and where credit cards are involved, that merchants have a right to protect themselves against fraud, etc.

[samuelweiler]

@marcoscaceres We disagree re: reasonable expectations, transparency, and negotiation. e.g. in the US, sometimes city is sufficient to determine tax. Perhaps even just state is enough - there is no need for specific street info. (And, also in the US, I tend to see sales tax determined by "shipping address", when something is being shipped.) There is lots of grey area here, and the API described is not modeling the reality well. And it's shipping more information than is needed in all cases.

[ianbjacobs]

Hi @samuelweiler, can you say more what you mean by this: "And it's shipping more information than is needed in all cases."

[marcoscaceres]

there is no need for specific street info.

Precisely why street info is removed by the redactList. Perhaps something that was overlooked during your review?

There is lots of grey area here, and the API described is not modeling the reality well.

Respectfully, I find this quite baffling a statement to make. We are literally working with a global set of merchants, credit companies, and other financial tech representatives in this working group for years to come up with the design.

Additionally, the API and precursors (e.g., Apple Pay) have been shipping in browser for over 2-3+ years - proving themselves adequate around the world for the purpose of purchasing both digital and physical goods. None of the membership or implemeters have come back and raised the address structures and contents as “not modelling reality well”?

I also feel the discussion is getting off track a little. We need to assume the shipping and billing addresses do model reality: but what is of importance to the discussion is limiting what information of the address is exposed. From our research and imput from the WG members that what we are redacting strikes the right balance - and we allow user agents to go further, such as user agent further redacting post codes to further protect user privacy.

[marcoscaceres]

@samuelweiler, my apologies, in my mind I thought we'd merged the following already (which redacts the addressLine): https://github.com/w3c/payment-method-basic-card/pull/67/files

I've also made the redactList mandatory: https://github.com/w3c/payment-request/pull/846

Does the above alleviate your concerns?

[samuelweiler]

Restating my concern, which will hopefully answer @marcoscaceres and @ianbjacobs's questions.

I am concerned about "oversharing" - when an API, by default, automatically sends more identifying information than is sent now. This API is sending straight up PII (personally identifiable information). Because of that, it should meet a high bar.

For the two use cases presented - calculating sales tax and shipping costs before completion of a transaction - this API overshares in some cases.

Below are some examples that speak to what information is minimally necessary for the use cases. I could also point at specific merchants that ask for less info (since my concern is about what this API sends v. what is sent now), but looking at necessity of the information is probably more helpful in understanding the problem.

• In the US, billing address is typically not needed for sales tax calculations - those are based entirely on shipping address. Sending any portion of a billing address - at least pre-completion - is oversharing.

• In New Hampshire, Delaware, Oregon, Montana, and Alaska - US states which have no sales tax - the only portion of a shipping address needed for sales tax calculation is the state. Sending city or post code pre-completion is oversharing.

• Also in the US, shipping costs are often flat. For the shipping use case, then, sending a city, post code, and even state pre-completion might be oversharing.

• Some US merchants offer flat shipping rates to the "lower 48". All these need to calculate shipping cost is "US" and "not Alaska or Hawaii". Post code and city are oversharing, and arguably even the specific state is oversharing.

These examples are all from the US - a thriving market for online commerce. And yet even with the mitigations in place so far, this API still overshares for these US regions. I imagine that similar oversharing happens when we look at other jurisdictions. What portion of which address is needed to calculate shipping cost or sales tax in the Cayman Islands? Do those calculations need anything more than the country? What about in Kenya?

Again, I want to avoid a default of oversharing - I want to avoid moving the bar so that more PII is sent 1) than is currently sent and 2) than is necessary. I don't entirely know how to satisfy your use cases under that constraint, but the diversity of tax and shipping price practices across jurisdictions suggests that a fixed answer (e.g. send city and post code) will overshare for a non-trivial number of people.

[ianbjacobs]

Hi @samuelweiler,

Thank you for the writeup.

According to the specs today, the merchant only receives billing address information through onpaymentmethodchange when requestBillingAddress is true. That information is redacted, as we've mentioned.

Thus, for some of the use cases you mention, where the merchant doesn't need any billing address information at all, that is the default behavior of the API. Merchants would explicitly need to set requestBillingAddress to true.

Ian

[1] https://w3c.github.io/payment-request/#dom-paymentoptions-requestbillingaddress

[asmusf]

I see two scenarios. One is the case of flat shipping rates and similar where merchant needs to determine only whether a location is in/out a larger area. Unless it's possible to make a request that specifically recognizes an area defined for shipping tariff purposes, it would seem the best way would be to be able to identify the most generic of standard postal address elements needed to make the decision in/out on the receiving end. Inside the US that would be state. The other case is where the granularity of the information required depends on the location. That's the case of not needing address details below state level only for those states where sales tax is zero. Unless the API is iterative ("I see this is in NY, please let me know the city") I don't see how this can be realized without building a lot of knowledge into the API.

[duerst]

I agree with Asmus. In addition, building the knowledge into the API means that there have to be regular updates, because tax laws and rates, and shipping costs,... can change.

Also for shipping costs, it may depend on the shipping company. For example, in Japan, shipping by postal service would be the same rate for the whole country, whereas some other company may charge more for far-off islands. Or at least the delivery date/time will be totally different. And while some of these islands can be distinguished at the prefecture level (i.e. Okinawa), others are part of the Metropolitan Area of Tokyo even though they are a few hundred kilometers to the south. On the other hand, taxes are the same all over Japan.

And then there are countries where some parts are half around the world from the main part. Think about Guam,... for the US, or Réunion,... for France.

[asmusf]

Seems the only way to design an API that strictly limits sharing in this kind of situation is to default to no information being sent by default, with the caller iteratively requesting more information items on an "as needed" basis, based on whatever partial information returned the previous round.

[samuelweiler]

@asmusf @duerst I agree that any static definition of what is sent (and what is redacted) will likely work very poorly. (Hence me choosing a title for this issue ("Richer negotiation...") that hints at a non-static solution.) I don't know how much value there is in explaining why certain elements are requested, but I encourage you to consider that, too.

[asmusf]

@samuelweiler You write: " I don't know how much value there is in explaining why certain elements are requested". Do you mean as part of the API? Or as part of the specification? Ideally, at all stages of a transaction, if personal information is being shared, the requester should be on the hook to get consent from the user. However, there's also a signal to noise ratio.

[samuelweiler]

@asmusf I mean as part of the API .

[ianbjacobs]

Hi all,

This issue relates most directly to Basic Card, so I have opened a new issue there: https://github.com/w3c/payment-method-basic-card/issues/72

Closing this one here for now. Ian

[pes10k]

I wanted to second the point made by @samuelweiler. We shouldn't be pushing / standardizing any functionality that, were a site to use it, would require them to increase shared information (especially not given the sensitivity of this information).

Addressing in 1, and not pushing back to 1.1 seems correct, before vendors start baking in expectations and web compat issues already start creeping in

[marcoscaceres]

Ok, let's address this in next round on CR (starting next week)? However, we will still publish an updated CR on TR... the one there is grossly out of date and long overdue for a refresh.

[ianbjacobs]

Hi @marcoscaceres,

I am reclosing this issue because we have opened a Basic Card issue for this topic: https://github.com/w3c/payment-method-basic-card/issues/72

I very much encourage brainstorming there.

Before re-opening, please let's discuss at the face-to-face meeting. That is only a couple of days from now.

Thank you,

Ian

[ianbjacobs]

Re-opening as part of discussion to go to Proposed Rec.

[pes10k]

Hi all, I was just wondering if there was an update on this issue, or if the WG could provide an update of where things stand. thanks much!

[EricMwobobia]

Great brainstorming on how to address the excess PII information . Any success in implementation? privacy wise this should be a great win for the end-users (customers paying via this feature).

[ianbjacobs]

Hi @pes10K,

I believe there has been no change to this issue. I am not aware of implementation experience for pull request #873, nor plans to implement. This was discussed briefly at the WPWG's 17 September call: https://www.w3.org/2020/09/17-wpwg-minutes#item03

Ian

[Sauski]

I think it’s worth revisiting this, to see if the changes proposed in PR #873 (support for requesting the entire address directly however removed) can be included in the earlier version of the specification.

It feels wrong to introduce a spec that reduces how good a good actor can be over the current state. If a site previously made the decision to limit the address data it attempted to collect then we should be supporting that behavior and enabling it to continue.

In concert with this, absent any well defined, UA managed, incentives for sites to request less than the maximum possible information, it seems that the spec should aim to make the most private option the path of least resistance.

Introducing an intermediate state that is the combination of the least effort and least private option, then later attempting to persuade the ecosystem away from that position by offering a higher effort & higher privacy option, seems more challenging than simply avoiding offering the possibility in the first place.

[samuelweiler]

@Sauski writes:

It feels wrong to introduce a spec that reduces how good a good actor can be over the current state.

The phrase "gratuitous harm to privacy" comes to mind. Thank you for giving this more voice, @Sauski.

[ricardo]

Hi everyone.

I thought I'd just add another use case here, because this seems to impact a lot of merchants using our WooCommerce Apple Pay implementation right now.

Currently, Apple Pay provides the merchants with only the first three characters of the ZIP code in the UK and Canada (e.g "SK1") when calculating the shipping methods. - The problem is that some merchants have wildcard postcodes (e.g "SK11*") with a minimum of 4 characters to display the available shipping methods. Redacting the postal code, in this case, breaks the checkout process for them.

It would be really nice to have the option to negotiate the postal code from being redacted.

[duerst]

@marcoscaceres Any information on why this was closed?

[ianbjacobs]

Hi @duerst,

We removed support for addresses in the API (itself). Addresses may still be supported in private payment method data, but there are no longer any explicit API features for them.

[samuelweiler]

... Addresses may still be supported in private payment method data, but there are no longer any explicit API features for them.

Pushing addresses out of the standard and into something not standardized is not a particularly satisfying answer - the privacy harm (potentially) remains the same, even though it has moved. That said, with it removed from this spec, I concur that this issue should be closed.

[ianbjacobs]

This issue was raised in Payment Request, but closed once we removed addresses from that API. We anticipate adding addresses back to the specification and so are re-opening this issue proactively.

[ianbjacobs]

We added addresses back to the specification in the 7 August 2024 CRD