CDR-API-Stream commented 2 years ago

Description

Issue #444 resulted in the new API GetDataHolderBrandsSummary being added to the Register APIs section of the Standards. This issue documented two solutions, an interim and a complete solution, to be delivered sequentially.

This issue has been raised to track the work required to document GetDataHolderBrandsSummary V2 in the Standards and the timeline to deliver to the ecosystem.

Area Affected

Register APIs > Get Data Holder Brands Summary

Change Proposed

Add V2 of the GetDataHolderBrandsSummary API where dataHolderBrandId becomes a mandatory field and the end state of the interimId is planned.

perlboy commented 2 years ago

For people looking for a solution immediately (@af-stayorgo) there is an undocumented version of public uri endpoints already available on the Register here: https://api.cdr.gov.au/cdr-register/v1/banking/register

@CDR-API-Stream this API appears to be in violation of the Extensibility guidelines of the Standards for the following:

This mechanism MUST NOT be used to create modified duplicates of the end points defined in the API Standards The end points in this area MUST comply with the standard's conventions and principles including naming conventions and data types. Data holders seeking to extend the standards MUST nominate a prefix to identify all extensions. New fields MUST comply with the naming conventions and data type standards used.

af-stayorgo commented 1 year ago

@CDR-API-Stream its great to see the GetDataHolderBrandsSummary API up and running.

Some of the responses from the published base URIs are returning empty data, for example Virgin Money - their API is returning a 200 response, however the payload is empty.

nils-work commented 1 year ago

For reference, linking the Register issue related to the delivery of the Get Data Holder Brands Summary endpoint - https://github.com/ConsumerDataStandardsAustralia/register/issues/187

ghost commented 1 year ago

For people looking for a solution immediately (@af-stayorgo) there is an undocumented version of public uri endpoints already available on the Register here: https://api.cdr.gov.au/cdr-register/v1/banking/register

@CDR-API-Stream Thanks for getting the GetDataHolderBrandsSummary up & running. It appears to be returning a subset of the information available in https://api.cdr.gov.au/cdr-register/v1/banking/register . Is there a plan to update GetDataHolderBrandsSummary to include the other data points such as website, cdrPolicyUrl, email, etc?

Eg, from https://api.cdr.gov.au/cdr-register/v1/banking/register:

{
            "legalEntityId": "f017aa48-2fc8-eb11-a83b-000d3a8842e1",
            "legalEntityRef": "DHBNK000195",
            "legalEntityName": "PayPal Australia Pty Limited",
            "industry": "banking",
            "emailAddress": "aucdr@paypal.com",
            "serviceAddressStreetAddress1": "Level 24, 1 York Street",
            "serviceAddresSuburb": "Sydney",
            "serviceAddresState": "NSW",
            "serviceAddressPostcode": "2000",
            "website": "https://www.paypal.com",
            "cdrPolicyUrl": "https://www.paypalobjects.com/marketing/ua/pdf/AU/en/cdr-policy.pdf",
            "abn": "93111195389",
            "accreditationDate": "2021-06-08T07:58:18Z",
            "logoUrl": "https://newsroom.au.paypal-corp.com/image/pp_h_rgb.jpg",
            "brands": [
                {
                    "brandRef": "000309",
                    "brandName": "PayPal Australia",
                    "brandDescription": "PayPal Australia",
                    "industry": "banking",
                    "website": "https://www.paypal.com",
                    "cdrPolicyUrl": "https://www.paypalobjects.com/marketing/ua/pdf/AU/en/cdr-policy.pdf",
                    "logoUrl": "https://newsroom.au.paypal-corp.com/image/pp_h_rgb.jpg",
                    "softwareProducts": [],
                    "status": "ACTIVE",
                    "participantType": "Data Holder",
                    "productReferenceDataApi": null
                }
            ],
            "legalEntityAssociations": [],
            "status": "ACTIVE",
            "lastUpdated": "2022-09-12T04:14:11Z",
            "participantType": "Data Holder"
        }

From GetDataHolderBrandsSummary:

{
            "dataHolderBrandId": "ebcbaeb6-539f-ec11-a82c-000d3a8830d6",
            "brandName": "PayPal Australia",
            "industries": [
                "banking"
            ],
            "logoUri": "https://newsroom.au.paypal-corp.com/image/pp_h_rgb.jpg",
            "publicBaseUri": "https://api.paypal.com/v1/identity",
            "abn": "93111195389",
            "acn": "111195389",
            "lastUpdated": "2022-09-12T04:14:11Z"
        }

af-stayorgo commented 1 year ago

@CDR-API-Stream we have just noticed that https://api.cdr.gov.au/cdr-register/v1/banking/register is currently returning 'null' for all 'productReferenceDataApi'. Can this be fixed?

ACCC-CDR commented 1 year ago

Hi @af-stayorgo, https://api.cdr.gov.au/cdr-register/v1/banking/register is solely intended for use by the cdr.gov.au website to populate the data holder and data recipient information. We have recently removed the data within the 'productReferenceDataApi' field in favour of pointing people towards the Get Data Holder Brands Summary API.

jxeeno commented 1 year ago

@ACCC-CDR Do we have a way of discovering the product data (EME CDR) endpoints for all energy brands? Currently, the Get Data Holder Brands Summary API is only returning consumer data sharing API endpoints for energy where the brand is not an active data holder (i.e. not providing consumer data endpoints). Once a brand becomes an active data holder, it returns the consumer data sharing endpoints which don't return product data.

That means active DHs - i.e. Origin, AGL and in the near future, Energy Australia - will not have publicly discoverable endpoints for product data.

nils-work commented 1 year ago

Hi @jxeeno

The Get Data Holder Brands Summary API includes a publicBaseUri for each Energy brand, but for ACTIVE holders, that endpoint is currently only relevant for finding the public Status and Outages endpoints that are required for their consumer data sharing.

For holders that are not yet ACTIVE, the publicBaseUri will be directed to their public Plan data, but not Status and Outages, as those are not yet applicable.

Hope that makes sense.

You can find more information about Energy PRD on the CDR website - How to find a data holder’s product data request service. That page provides a link to AER - Energy product reference data, which provides a link to -

List of energy retailer base URIs

The PDF on that page provides a full list of URIs for all retailers currently hosted by AER. An alternative is the datasources.json file used by the CDS Product Comparator demo which is a JSON snapshot.

The above options may be considered transitional, as the live Register API should become the correct source over time.

jxeeno commented 1 year ago

Yep, that explanation makes sense. Having the source of truth as a PDF is less than ideal, but thank you for pointing out datasources.json as something we can use in the interim @nils-work

I guess my question now is whether the work to update the Register API to return energy PRD endpoints regardless of DH status is something that's being tracked somewhere or if it's a separate issue that needs to be raised?

My read of https://github.com/ConsumerDataStandardsAustralia/standards-maintenance/issues/444 and https://github.com/ConsumerDataStandardsAustralia/standards/issues/237 is that the interim v1 solution was intended to deliver the ability for unauthenticated users to call GetDataHolderBrandsSummary API v1 to get the PRD, outage and status endpoints. It seems like neither the current v1 interim solution nor the current v2 scope quite delivers on this for energy.

nils-work commented 1 year ago

Hi @jxeeno

The issue you've described is due more to the Energy sector designation, current rules, and Register onboarding process (timing), than the Summary API itself.

The API may return a single publicBaseUri for each brand, but the endpoints available at that base will vary over time due to the reasons above. This is a known situation and any solutions may not involve a change to that API.

The CDR Register (which is the source for the Summary API) currently has a single field for a brand's publicBaseUri
In banking for example, the single publicBaseUri is generally expected to cover two different types of public endpoints - PRD, and also Status and Outages
For energy retailers that are currently INACTIVE in CDR, their base points to the PRD hosted by AER/Energy Made Easy (EME) as those retailers don't require Status and Outages yet (though they appear to be supported by EME, for PRD)
For retailers that become ACTIVE in CDR, they are required to share consumer data. This requires the Status and Outages endpoints to be available to provide information on the consumer data sharing services. In this situation, the base would be expected to point to the retailer's Status and Outages platform, even though EME also continue to host the PRD. The challenge in this case is having one base URI in the register that may be expected to provide for two separate sets of public endpoints. That was part of the discussion in issue 248.

jxeeno commented 1 year ago

Great, thanks for sharing the additional context @nils-work. Will continue the discussion over here

ConsumerDataStandardsAustralia / standards-maintenance

Deliver V2 of the GetDataHolderBrandsSummary API #518

Description

Area Affected

Change Proposed

List of energy retailer base URIs