Add entity resolution capability to search

[steveburtch]

To improve the quality of our search results, we need to be able to resolve entities so that we can collapse multiple entries into a single one based on a set of rules.

Business Analysis

• [x] Initial meeting of an entity resolution working group to frame the problem and explore lines of investigation

Meeting output: See comments section below.

[steveburtch]

Initial invitation list: Steve B, Thor, Avinash, Argus, Kial.

[steveburtch]

scheduled kick-off mtg for Fri Jun 2

[Apt766525]

Meeting output: Every company that needs to do taxation registers with the CRA and the company is given a BN9 number. And for every program you are in, they give an additional 6 characters to the base BN9 that gives the BN15. For every program registered they get different BN15 but the base BN9 is the same one. In our case we can use the BN9 number for verifying if it’s the same person.

[steveburtch]

Will create a spike ticket to initiate investigation into how we can use the BN9 for initial verification.

[mstanton1]

As discussed this entails the ability to collapse multiple rows for a business into one in search. A concern raised is that the search shows more than just significant individual information, and the positions an individual holds in relation to each company can vary. Design wise, confusion may arise with collapsing.

For individuals, a profile would provide a mechanism to collapse / confirm the individual listed as an SI on one record is the same individual listed as an SI on another record.

@mstanton1 To discuss with @kialj876 / @thorwolpert to understand if this would still add value. At a minimum, this appears to be something that could be tackled post mvp.

[thorwolpert]

@mstanton1 this ERO one is to figure out if we can collapse people. Eg. In Colin, and now Lear, people are added uniquely per registration. So we have a person, same name and same address like 50 times in there, so can we collapse them into 1 person??

[mstanton1]

Ah, @thorwolpert is that collapsing people on the backend (e.g. some sort of guid or identifier in the DB for a person rather than having the same full record multiple times) or is this related to the visual display of results for a search? If it's within the DB is this something we'd be able to look at ahead of implementing a profile concept in order to prepare?

[thorwolpert]

Many months ago we discussed lots of options, I don't think any were expanded on. It's a known issue that we'll have pages & pages of very similar looking data. If we want to try and collapse them, we'll need to to that in the analytics database as Search wont be able to collapse them on its own.