HUD-Data-Lab
commented
3 months ago Draft language:
Deduplication
As outlined in the HMIS Data Dictionary, it is critical that HMIS and comparable database implementations have the capability to deduplicate client records. Whenever possible, it is recommended that there is a 1:1 relationship between an individual and a Personal ID. If duplicate records are created, they should be merged into a single record unless this would violate a privacy constraint.
Regardless of whether an individual has multiple records in the system, internal deduplication for the purposes of the Personal Identifier ensures that they are only counted once in reporting. External deduplication is needed in data integration projects to ensure that clients in the incoming data set who already have a record in the receiving system see the new information added to their existing record, rather than having an entirely new Personal ID assigned to them for the incoming data.
There is significant flexibility in implementing deduplication algorithms. Possible deduplication schemas include, but are by no means limited to, the following:
Schema 1
Exact match on:
- Name
- Social Security Number
- Date of Birth
- Race and Ethnicity
- Gender
- Veteran Status
Schema 2
Exact match on:
- Last four digits of Social Security Number
- Date of Birth
Fuzzy match on:
- Name
Exact match on two or more of:
- Race and Ethnicity
- Gender
- Veteran Status
Schema 3
Exact match on:
- Name
- Date of Birth
AND EITHER:
-
Exact match on:
- Last four digits of Social Security Number
OR
- Social Security Number is null AND
- Perfect match on two or more of:
- Race and Ethnicity
- Gender
- Veteran Status
Vendors are encouraged to choose a robust deduplication method and to be transparent with their communities about how this deduplication is performed.
eric-jahn-bitfocus
eanders
kaungst
Request for Comments: Client Deduplication API
Summary
This RFC proposes the addition of a new REST endpoint to help deduplicate client records. The endpoint will take a client's first name, last name, date of birth, and social security number as inputs and return a client ID. This client ID will either be an existing client ID if a match is found, or a new client will be created and its ID will be provided for future requests.
Motivation
Non-HMIS vendors do not have a standard means for confirming the existence of an individual within a HMIS. To decrease the liklihood of duplicate records, this endpoint provides common input across HMIS for CoC vendors to confirm existence of and pull records related to a homeless individual.
Specification
Endpoint
POST /clientsRequest
The request body should contain the following fields:
firstName(string, required): The client's first name.lastName(string, required): The client's last name.dateOfBirth(string, required): The client's date of birth in YYYY-MM-DD format.socialSecurityNumber(string, required): The client's social security number.Example:
Response
The response will return a client ID, either an existing client ID if a match is found or a new partial client ID if no match exists.
Error Handling
The API should handle the following error scenarios: