Revamp matching algorithm

thinkingmachines / linksight-2018

LinkSight is a web app for applying the Philippine Standard Geographic Code to messy and misspelled barangay, municipality, city, and province names.

https://linksight.thinkingmachin.es

GNU General Public License v3.0

11 stars 1 forks source link

Closed stuckoverflo closed 6 years ago

stuckoverflo commented 6 years ago

Suggestions:

split the PSGC code into their own parts
for matching, use recordlinkage first, then use fuzzywuzzy for parts where we don't see an easy match
make the interlevels generic for usability
also look into dedupe

User acceptance test

[x] Algorithm can process 100 rows in under 15 seconds
[x] Algorithm can handle datasets that are missing barangay, municipality/city, or province (any combination of these three levels)

marksteve commented 6 years ago

I think @liangi meant exact matches first then fuzzywuzzy?