Closed iibarant closed 2 years ago
Hi @iibarant
I would think the latter approach is better and much faster because it finds only those matches between the single string in the new Series and the strings in the other larger Series.
The issue with the former approach is that it also includes matches between all strings in the Series which is not what you are looking for and therefore inefficient.
By the way, regarding the California data you mentioned some time back, you could consider using the regex
option of red_string_grouper (or string_grouper) to remove those common abbreviations from the strings in the Address field before performing the comparison. Then the matching would proceed much faster and there would be no matching based on those common abbreviations.
Hi @ParticularMiner,
That works perfectly. Thank you. A question - a tester put completely different address with only state, city and zip code matching and similarity = 0 just to test expecting many records to show up. The procedure returned empty data frame. Why?
Hi @iibarant
Sorry I did not understand your last question.
Did you mean: someone entered a new string in the address field. This string had state, city and zip code information which was expected to match many records. But nothing showed up?
If so, how large was the entire DataFrame and was the threshold similarity value (min_similarity) low enough?
Yes, you got it right. The dataframe is much smaller - around 7 thousand records, the min_similarity was set to 0.
On Sep 29, 2021, at 1:37 PM, ParticularMiner @.***> wrote:
Hi @iibarant
Sorry I did not understand your last question.
Did you mean: someone entered a new string in the address field. This string had state, city and zip code information which was expected to match many records. But nothing showed up?
If so, how large was the entire DataFrame and was the threshold similarity value (min_similarity) low enough?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
@iibarant
And you used the original match_strings()
function from string_grouper
? Were any other options used?
@iibarant
So the call was something like this:
match_strings(df['address'], df_with_one_new_string['address'], min_similarity=0) ?
I found a global variable that was used on my side and kept the similarity high enough. Looks good now.
Hi @ParticularMiner,
I hope you are doing good. Imagine I have an address column in a dataframe and I want to find the best matches from it to an address that I enter in a query. One approach would be to add an extra row in the dataframe with the address field from the query and run the standard match_string. Another approach would be to create a dataframe with one row and match the 2 with match_string with an extra df.
What would you suggest?
I will be trying to play in both directions.
Thank you!