Closed ronaldexim closed 3 years ago
The results are actually a feature and not a bug :sweat_smile:
Under the hood, the grouping is actually done using a TF-IDF instance instead of the embeddings that you passed. If you want to group the results with your embeddings, you will have to pass your BERT
model to group
:
model.group(bert, link_min_similarity=0.75, group_all_strings=True)
And that's it! I use TF-IDF as default as any embedding or edit distance technique often takes quite a while and I figured that you would want more speed in the group
function as a default. You can use any different technique for grouping if you would like.
Thank you - it works great now.
I tried with model_id "BERT" but it should be polyfuzz.models.Embeddings
- 'bert'.
Awesome library :clap:
If I work with tf-idf it is fine:
{1: ['apples', 'apple', 'appl']}
but with embeddings:
{}
same with
bert-base-multilingual-cased
installed withpip install polyfuzz[flair]