search

miohtama / python-Levenshtein

The Levenshtein Python C extension module contains functions for fast computation of Levenshtein distance and string similarity
GNU General Public License v2.0
382 stars 229 forks source link

get_matching_blocks() does not align with ratio() #12

Open ghost opened 2 years ago

ghost commented 2 years ago

image

Example: For strings "abc" and "dac", get_matching_blocks() gives (2, 2, 1), (3, 3, 0) as matching blocks, that is, the substrings "c" and "" from the two strings.

Based on the matching blocks (2, 2, 1), (3, 3, 0) returned by get_matching_blocks(), the Levenshtein ratio would be 2 (1 + 0) / (3 + 3) = 0.333... However, the ratio() is 0.666... In fact ratio() should be the correct one, because in the sense of "minimal string distance" we should indeed regard "a" and "c" as the two best matching blocks, yielding 2 (1 + 1) / (3 + 3) = 0.666... So why doesn't get_matching_blocks() capture "a", i.e. (0, 1, 1), also as a matching block?

It's sad to learn that this library is currently not being maintained. I just leave this open quesition here and hope for someone's answer in maybe 2030 :)

maxbachmann commented 2 years ago

ratio and get_matching_blocks use two different algorithms. ratio is based on the Indel distance, which only allows Insertions and Deletions, while get_matching_blocks is based on the normal Levenshtein distance.

In fact ratio() should be the correct one, because in the sense of "minimal string distance" we should indeed regard "a" and "c" as the two best matching blocks, yielding 2 * (1 + 1) / (3 + 3) = 0.66

get_matching_blocks simply does no guarantee, that it will behave this way. In you case the edit distance is 2. However there are multiple ways to convert them accordingly: 1) "abc" -> "dbc" -> "dac" 2) "abc" -> "dabc" -> "dac"

You expect it to take the second path. However it will simply take one of them (scanning all possible paths is a lot slower)

It's sad to learn that this library is currently not being maintained. I just leave this open quesition here and hope for someone's answer in maybe 2030 :)

This library has not been maintained for a long time. I am however actively maintaining a fork of this library: https://github.com/maxbachmann/Levenshtein. If you expect a drop in replacement for difflib, which has the corresponding behavior for get_matching_blocks, you can use https://github.com/maxbachmann/CyDifflib instead.