RESOStandards / transport

RESO Transport Workgroup - Specifications and Change Proposals
https://transport.reso.org
Other
18 stars 15 forks source link

[RCP-35] Single Feeds #96

Open darnjo opened 11 months ago

darnjo commented 11 months ago

Discussed in https://github.com/RESOStandards/transport/discussions/71

Originally posted by **EnFinlay** January 27, 2023 This post is a follow up from [RCP 035](https://reso.atlassian.net/wiki/spaces/RESOWebAPIRCP/pages/8417574946/RCP+-+WEBAPI-035+Usage+Resource+for+Field+Usage+Permissions). The goal of RCP-035 was to propose a new resource that could be used to help meet the needs of the community by including metadata as data (the Usage resource) to communicate the permissions and allowed usage for each field sent to a data consumer. I would like to withdraw that RCP or heavily revise it as proposed below. **Synopsis** A new field called `FeedType` of type Multi Enum String should be included in every data resource and the fields resource. On data resources, this field will be used by the data provider to indicate which feed included the given record. On the fields resource (which is a metadata resource), this field will be used by the data provider to indicate whether the given field is included in a given feed type. **Business Case** A business case for this is a data consumer with multiple feeds from a single data source, but each feed has different usage restrictions. This proposal would allow the data consumer to only ingest a single feed, but still have enough information from the data to use records and fields appropriately in the various contexts they are operating in. **Example** Hypothetically a data consumer could have two feeds from an MLS - IDX and BBO. The IDX feed contains only Active records, and of those records it only includes the `ListPrice`, `StandardStatus` and `UnparsedAddress` fields. The BBO feed contains only Closed records, and of those records it only includes the `ListPrice`, `StandardStatus`, `UnparsedAddress`, and `CloseDate` fields. Currently the data consumer would have two feeds: Feed 1: ``` [{ "ListPrice": 1000, "UnparsedAddress": "123 Main Street", "StandardStatus": "Active" }, { "ListPrice": 2000, "UnparsedAddress": "100 Swanky Street", "StandardStatus": "Active" }, ... ] ``` Feed 2: ``` [{ "ListPrice": 11111, "UnparsedAddress": "400 12th Ave", "CloseDate": "2020-01-01", "StandardStatus": "Closed" }, { "ListPrice": 22222, "UnparsedAddress": "777 Garden Road", "CloseDate": "1998-02-03", "StandardStatus": "Closed" },{ "ListPrice": 1000, "UnparsedAddress": "123 Main Street", "CloseDate": null, "StandardStatus": "Active" }, { "ListPrice": 2000, "UnparsedAddress": "100 Swanky Street", "CloseDate": null, "StandardStatus": "Active" }, ... ] ``` This proposal would present the data as a single feed that would look like this: ``` [{ "ListPrice": 1000, "UnparsedAddress": "123 Main Street", "CloseDate": null, "StandardStatus": "Active", "FeedType": ["IDX", "BBO"] }, { "ListPrice": 2000, "UnparsedAddress": "100 Swanky Street", "CloseDate": null, "StandardStatus": "Active", "FeedType": ["IDX", "BBO"] },{ "ListPrice": 11111, "UnparsedAddress": "400 12th Ave", "CloseDate": "2020-01-01", "StandardStatus": "Closed", "FeedType": ["BBO"] }, { "ListPrice": 22222, "UnparsedAddress": "777 Garden Road", "CloseDate": "1998-02-03", "StandardStatus": "Closed", "FeedType": ["BBO"] }, ... ] ``` and the Fields Resource would look like this: ``` [{ "FieldName": "ListPrice", "FeedType": ["IDX", "BBO"] }, { "FieldName": "StandardStatus", "FeedType": ["IDX", "BBO"] },{ "FieldName": "UnparsedAddress", "FeedType": ["IDX", "BBO"] },{ "FieldName": "CloseDate", "FeedType": ["BBO"] }, ``` Given this setup, the data consumer knows that every data record with the "IDX" feed type can be used for "IDX" purposes, and that records with the "BBO" feed type can only be used for "BBO" purposes. Additionally, given the requirement that every record has the same schema, the "Fields" resource comunicates that even though the "CloseDate" field is present on an "IDX" record, that field cannot be used, since that record is not part of the "BBO" feed. Data providers could `null` out the value of the fields that the data consumer does not have permission to use. **Proposal** **New Field on Every Data Resource** A new field called `FeedType` of type Multi Enum String will denote which feeds a given record is included in. The values of this field will correspond to different feeds that a data consumer is approved for. These feed names do not necessarily need to be human readable, although it's important that data provider and consumer agree on the allowed usage of each feed. **New Field in Fields Resource** A new field called `FeedType` of type Multi Enum String will denote which fields are included in a given feed. This field will have the same lookup values as the `FeedType` field on the data resources.