Previously, _settoken ratio, through a mixture of join and split and strip concatenated all tokens together with no whitespace. This allowed for partial matches across token boundaries. This can occur in practice when a human enters a search, but is rare.
Change: Implement Levenshtein's setratio() scoring and preserve tokenization in fuzz._set_token_ratio
Now fails 2 tests due to score changes, which should be expected.
testTokenSetRatio: score improves
testWithCutOff: score improves to above 50
Previously, _settoken ratio, through a mixture of join and split and strip concatenated all tokens together with no whitespace. This allowed for partial matches across token boundaries. This can occur in practice when a human enters a search, but is rare.
Change: Implement Levenshtein's setratio() scoring and preserve tokenization in fuzz._set_token_ratio
Now fails 2 tests due to score changes, which should be expected.
testTokenSetRatio: score improves testWithCutOff: score improves to above 50