The Data Holder PVT Problem

[rob-hale]

Description

The ability to test if DH endpoints are functioning correctly is an important part of CDR. This capability is needed by DHs to conduct Production Verification Testing (PVT), particularly following change or maintenance activity. It is also needed by ADRs and intermediaries in order to troubleshoot issues raised by consumers. The ACCC also arguably should be able to determine endpoint performance as part of enforcement activities. This situation exists globally, it is not unique to Australia.

What is unique to Australia, is the CDR. Now that CDR includes Energy, PVT becomes more problematic. When it reaches into Superannuation and Insurance it will become even more problematic. This problem needs addressing now.

Note: Discussion touched on this previously when discussing ADR Metrics - https://github.com/ConsumerDataStandardsAustralia/standards/issues/240

Area Affected

To date in banking, the approach for ADRs has been to create production accounts with each bank. These accounts are then used to test the services of that DH. Where OTPs are sent via SMS, ADRs also need to associate a mobile services to receive these OTPs. With a small ecosystem, this approach just about works, although there will already thousands of these 'test' accounts in existence. Each one is associated with a real consumer who has had to undergo increasingly challenging AML/KYC processes imposed by each DH. Creating and managing these accounts is costly for ADRs and DHs.

While this approach works for mainstream bank accounts and simple use cases, it cannot accommodate every nuanced configuration that consumers may encounter. An issue that only becomes apparent with a specific type of account, configured in a certain way, is unlikely to align with test accounts available to an ADR. Issues associated with Secondary Users and Nominated Reps are notoriously hard to troubleshoot for this reason.

Even if we turn a blind eye to this situation in Banking, we can't do the same in Energy. ADRs can't create new energy subscriptions resulting in electricity meters being installed on the homes of their employees. So they will have no way of testing their services and troubleshooting consumer issues. When Superannuation and Insurance come to CDR, ADRs aren't going create investment portfolios and insurance policies in this way either.

What happens when Action Initiation comes along?

Change Proposed

This matter requires careful thought and consideration, followed by action in order to ensure consumers receive value from CDR in these emerging sectors.

Two Possible Options

1. Create a centralised platform that has access to a set of accounts with each DH. Authorised participants would subscribe to services to conduct PVT or troubleshoot consumer issues. This reduces the overall number of accounts required.
2. DHs are required to create and maintain a series of test accounts for this purpose in production. These accounts could be configured for a broad set of use cases and test scenarios with associated test data.

Option 2 seems more challenging to do, but more useful and effective in the long term. It could also address the issue of imperfect conformance testing. With access to a carefully curated and consistent set of test data at each DH, the ACCC could enhance CTS for onboarding of new ADRs and software products. ADRs could also have more confidence that their solutions will perform as intended before releasing them to consumers.

For this to work, we'd have to overcome numerous regulatory issues, not least the KYC/AML obligations of DHs to identify and verify consumers. We'd also have to have some method of managing OTP and authentication requirements. Changes to legislation and rules will likely be required. This is all possible of course.

This appears to be an important piece of work for Australia's CDR community to take on - now

[markskript]

Skript supports this idea.

We acknowledge that option 2 will be challenging, as test accounts in live production systems are generally frowned upon. None the less - we too have had to signup as individuals with 40+ data holders just in order to test in Production. Ideally, passing the conformance testing would be enough for an ADR but the reality of the scenario is no two DH's have interpreted the standards in the same way so there is a need for ADR's to be able to test with individual DH's directly.

[jimbasiq]

Well done raising this @rob-hale

Basiq are also supportive of further investigation and working towards a proposed solution.

Basiq also have multiple "test" accounts with Data Holders. We use these regularly to verify stability of the platform (something we should not need to do).

We also have an issue with the DHs we have not been able to create "test" accounts for, either due to regional or industry restrictions in their membership requirements. For these smaller entities we simply have to wait until a consumer on our platform interacts with that DH to confirm a change is working - this is clearly undesirable.

When it comes to Energy this method of testing is even more of an issue as each individual can only have one service provider for each utility (unlike Banking DHs which have a one to many capability). This requires many "test" consumers on our side.

Regarding the comment "An issue that only becomes apparent with a specific type of account, configured in a certain way, is unlikely to align with test accounts available to an ADR. Issues associated with Secondary Users and Nominated Reps are notoriously hard to troubleshoot for this reason.". I would suggest a staged implementation. I would not want the DHs to be paralised by the number of test accounts they would need to creat to cover every possible type of member on their books. An early implementation covering most common member types would be a great start.

[Michelle-Suncorp]

Suncorp are also supportive of further investigation into what's possible to enable all participants (Data Holders and ADRs) greater functionality for production testing. Suncorp also request any solution designed consider enabling data holders (who may not yet have recipient capabilities) the ability to request and view data with test accounts with other data holders and/or themselves without reliance upon existing ADRs or their own DR solutions which do not always support all the required auth scopes and/or user profiles needed for production testing due adherence to the data minimisation principle.

[joshuanicholson]

SISS agrees with this Proposal. It would seem like many ADRs; we have set up test bank accounts across many of the larger national brands of Banks. We strongly agree with Rob, that the ability to create a bank account is relatively easy and has a mostly minor impact on all stakeholders. However, the ability to access the majority of banking products across all banks is not viable; how many home loans, credit cards, and business & personal accounts can one person or business have? We would hate for any of our staff to have negative personal consequences because they have 20, 40, 70 or more banking products across multiple institutions. This should not be the standard operating procedure for an ADR or its employees.

Like Rob, SISS is more concerned about future sectors; non-bank lending, Insurance, Superannuation, investments etc. We believe our ability to create 'test' accounts to be extremely limited or close to impossible.

With regard to the suggested options, there are pro's & con's in relation to both. Consideration should be given to the smaller DHs who don't have the resources to have the extra compliance, but SISS would prefer option 2.

Is there an option 3 here 3) DHs are required to create and maintain a series of test accounts in a pre-production environment (UAT). These accounts could be configured for a broad set of use cases and test scenarios with associated test data. This environment must be equivariant to the production environment. This allows DH to control access and users must agree to the DH T&Cs, to isolate legal or security risks.

[DougFromPayPal]

PayPal is supportive of further investigations into a viable solution enabling the ability and the mechanism for both Data Holders (DH) and Accredited Data Recipients (ADR) to perform Post Verification Testing (PVT). At this point in time, PayPal is solely classified as a Data Holder, but is supportive of participating in the discovery discussion from the perspective of both an ADR and a DH entity.

Presently, PayPal does not allow development teams to set up test accounts in the production environment, so any PVT activity is limited to technical monitoring and alerting features to determine the health of the production implementation. Although the CDR solution is heavily testing in lower environments, it is not ideal given the potential for environmental differences masking issues. It is also important, as a data holder, to participate in this testing solution to ensure we can help craft a solution that supports a testing facility while not overburdening the Data Holders with creating and maintaining thousands of test accounts.

As a Data Holder, it would be beneficial to have the ability to use an ACCC/DSB testing facility to test our production CDR solution to ensure it is working as expected and in compliance with the CDR regulation.