Closed JamesMBligh closed 5 years ago
Thanks for the informative session today, we all found the conversations useful. As we move to an iterative model of reviews we will provide our feedback incrementally too. One additional minor adjustment to the standards for your consideration:
OK. I'll consider an alternative that won't cause confusion.
-JB-
Various points of clarification would still be appreciated on this issue:
We should clarify that in our comments and the articles linked to on cursor based pagination that are not referring to the database cursor concept which indeed does require creation and violates a stateless and idempotent model. The cursors we are referring to are sometimes called continuation tokens and can indeed be implemented in a stateless and idempotent manner from a client perspective. We welcome the ability to implement this type of pagination through the first/next/prev/last links. However, the performance benefits of the cursor approach cannot always fully be realised unless the page, totalRecords and totalPages parts of the response are not included in responses. Perhaps the best compromise here is to require them only when a cursor hasn’t been provided. We also think it would be a good idea to allow data consumers to be able to request the maximum page size set by the standard for bulk transfers, so that if the standard is updated client code need not be modified. Wording suggestion for pagination that achieves this (additions in italics):
Pagination
Each API end point that can return multiple records will stipulate whether pagination is supported for the end point or not. For end points that will return less than a reasonably sized page of results in the majority of circumstances support for paging may not be included. Note that the use of paging for an end point does not require or preclude the use of filtering query parameters. It is expected that filtering and paging will be applied independently of each other. Note the intention here is a parameter set where the page parameter is absent and the page-size is set to “max” triggers a bulk-mode whereby a less functionally rich but more performant interaction model is triggered.
Query Parameters
The consumer will stipulate pagination requirements on the request using query parameters. When paging is supported the consumer MAY provide the following query parameters:
- page – the page number being requested (with the first page being 1)
- page-size – the number of records to return in each page. Set to “max” to retrieve the maximum allowable page size for this endpoint. If the query parameters are not provided the following defaults will be assumed:
- page – a default of 1 (the first page) will be assumed unless first/last/prev/next links have been followed and extra query parameters are present.
- page-size – a default of 25 will be assumed
Response Fields
In addition to the data requested a provider MUST provide the following additional information in the response payload:
- In the links object the following fields are to be provided:
- first - A URI to request the first page. This field MUST be present. The URI MAY include query parameters which are not included in the standard.
- last - A URI to request the last page. This field MUST be present unless there is only one page in the set or the page request parameter is not present and the page-size request parameter is set to max. The URI MAY include query parameters which are not included in the standard.
- prev - A URI to the previous page. This field MUST be present unless the current page is the first page or the page request parameter is not present and the page-size request parameter is set to max. The URI MAY include query parameters which are not included in the standard.
- next - A URI to the next page. This field MUST be present unless the current page is the final page. The URI MAY include query parameters which are not included in the standard.
- In the
meta
object the following fields are to be provided:
- totalRecords - The total number of records in the set. This field MUST be present unless the first/last/prev/next links have been followed and extra query parameters are present or the page request parameter is not present and the page-size request parameter is set to max.
- totalPages - The total number of pages in the set. This field MUST be present unless the first/last/prev/next links have been followed and extra query parameters are present or the page request parameter is not present and the page-size request parameter is set to max. For each of these fields the page size specified in the request should be assumed when calculating values.
Additional Pagination Rules
- Providers are not expected to implement pagination with transaction isolation. The underlying data-set may change between two subsequent requests. This may result in situations where the same transaction is returned on more than one page.
- A maximum page size of
1000
records is assumed for all end points (unless otherwise stipulated in the end point definition). If a page size greater than this maximum is requested then a HTTP status of422 Unprocessable Entity
SHOULD be returned.
This comment follows our earlier feedback on the draft Standards in relation to Loyalty Points. The Summary of Feedback on Draft Standards – Feedback Cycle 1 indicates that stakeholders had requested the ‘loyalty points balances and bonuses’ in the payloads, however Data61 noted that in their view this was not within the data set out in the designation instrument. To clarify, we were not proposing to include the ‘balance’ of rewards points account; which could either be held by the credit card provider for a loyalty program operated directly by the provider or by a third party, such as an airline, for external programs. Rather, we were proposing:
1) GET /common/customer - Prefix
This would be better optional, as it relates to mailing information (as opposed to directly tied to the customer) and thus would need to be derived from mailing information.
Name | Type | Required | Restrictions | Description |
---|---|---|---|---|
» prefix | string | true | none | Title or salutation. |
2) GET /common/customer - Occupation Code
Does ‘should’ below mean we don’t have to comply to the standard list? Ideally alignment to the standard list shouldn't be required since there will be differences where previous versions or other variations have been used.
Additionally we question whether this should be included in the customer end point, as it isn't fundamental to the customer and could be representing a point in time position (e.g. when a customer applied for loan).
Name | Type | Required | Restrictions | Description |
---|---|---|---|---|
»occupationCode | string | false | none | Value should be a valid ANZCO v1.2 Standard Occupation classification. |
3) GET /accounts/{accountId} - Account number in payload The recent feedback cycle has the comment from Data 61 that:
However we question if this (partially) negates some of the goals of ID permanence:
IDs MUST be immutable across sessions but MUST NOT be transferable across data consumers. For example, data consumer A obtaining an account ID would get a different result from data consumer B obtaining the ID for the same account even if the same customer authorised the access. Under this constraint IDs cannot be usefully transferred between client organisations or providers.
IDs MUST NOT be transferable between different customers for the same data consumer. For example, a data consumer should obtain a different ID for a joint account if the ID was obtained independently using authorisations from both customers.
Once the account number has been shared it will be much easier to derive the connections both between customers (i.e. joint accounts) and different Data Recipients customer data sets. So it would seem that if account number is to be included the above ID permanence rules should be reconsidered.
To note: at the in person feedback sessions (6-Dec-2018) it was noted by Data 61 that this account number may be required for payments, however if 'write' APIs were included in Open Banking then for payment initiation the AccountId could instead be used. It is true that for payments to the account from another payment provider, the account number could be useful.
As an existing consumer of data from the major Australian Banks, SISS Data Services would like to restate that not having a transaction type is reducing functionality as compared to existing data feeds.
SISS Data Services would recommend that the draft specification include a field called ProprietaryTransactionCode (string) which any Financial Institution can populate with their internal transaction code (if they have one). The majority of the data today is delivered via BAI2 files, and the Type Code field (which is required) is populated.
The draft specifications could also add a separate (optional) field ProprietaryTransactionCodeScheme (string). The Financial Institutions would populate this field with the scheme used (if any) - i.e. BAI2.
Having this information available means that Accounting Solutions (such as Intuit, MYOB, Sage) can further enhance the accuracy of rules for automatically coding records, reducing work that is fundamentally dead-time for Consumers and Small Businesses.
This field could also be used by other FinTech solutions to retrieve a reduced set of transactions relevant to their service. I.e. a PFM solution analysing amounts of recurring Automatic Payments.
Quick one on the swagger currently defined - just two typos
on definitions/Account - the accountid property should be accountId (so the required keyword works) the same is true of definitions/Balance
@WestpacOpenBanking - regarding versioning and swagger files - we currently don't know. We will seek to resolve this in 2019. It is likely that we will move to end point specific swagger files with one file for each version alongside a single swagger file containing the latest version of all end points but this is currently just a hypothesis.
-JB-
@WestpacOpenBanking - regarding pagination. I understand that idempotency can be maintained via cursors but this is not always the case and, as the feedback was linked to concerns around performance it was assumed that the overheads for maintaining idempotence would have equivalent performance issues. Even with idempotence support the position will for the standard will remain unchanged.
Wording indicating that cursors may be supported via links will be added to the pagination section. The suggested wording will be considered for this change. I am not supportive of the removal of the meta tags if cursors are used as the client would not necessarily be aware of the use of cursors and the use of the links does not necessitate the implementation of cursors. Also, if a continuation pattern is being used there is scope for the retention of the total record count and total page count via this pattern.
The suggestion regarding a request for max page size is interesting. I would be interested to hear from other members of the community on this.
-JB-
@MichaelARCA - regarding loyalty points.
The suggestion to include bonus loyalty points as a feature of a product definition will likely be included in the next round of changes to the product reference payloads as this is material to a customer making a product choice.
At this stage we are looking to include loyalty points in the transaction history due to concerns around the ability for banks to implement this although this would be a good item to consider as we develop a roadmap for what comes after the first implementation.
-JB-
@anzbankau - prefix is intended to be optional. This was a documentation defect that is being resolved.
-JB-
@anzbankau - for occupation code, the should will be removed from the description. If this data is not held in a standard form and cannot be mapped then the field should not be populated. The field is currently optional.
I was led to believe, however, that reporting of occupation code in ANZCO format was a recquirement for ATO reporting. If this is the case then it would be curious if a mapping couldn't be provided in at least some cases. I may have misunderstood the ATO requirements, however.
-JB-
@anzbankau - regarding account numbers, the ID permanence rules were to prevent the CDR regime inadvertently creating new ID types that could be used outside of the context of the APIs. These principles were not to preclude any type of unique identifier from the payloads. If this was the case we would not be able to include ACN, ABN, ARBN, Phone Numbers, PayIDs, BIC, APCA number, etc.
When the in person comments regarding using account numbers for payments were made I believe I specifically stated that these payments could occur through other channels. It is common practice for BSB/Account Number to be passed between entities to facilitate payments. The fact that banks retain registered payees containing this information is an example of this.
-JB-
@speedyps - regarding transaction type, a response on this topic was provided in the last round of feedback. A transaction type with a minimal set of standard types will be included. Individual banks can provide additional types if they wish using the extensibility mechanism.
Proposed values (which are open for feedback but will likely be what is published this week) are:
-JB-
@speedyps - thanks for the typos. These will be fixed in the next commit
-JB-
As the Christmas Draft version of the standards is imminent I will close this thread and create a new one to capture the next round of holistic feedback. If there is further comment on any of the responses above feel free to respond in the new thread.
-JB-
In decision proposal 39 a significant amount of feedback was received on the standards as a whole and a documented response to this feedback was provided. The standards site will now be incrementally updated to accommodate this feedback.
In the meantime this thread has been created to allow for more feedback to be provided, both in response to the outcomes of the previous feedback cycle but also to address any additional issues that have not yet been raised.
Note that a separate thread has been created for comments on product reference data. Please post feedback on this topic in that thread.
-JB-