Open user8114 opened 3 years ago
How many sentences are there per book?
100-400 sentences per book. Average around 180.
util.pytorch_cos_sim(embeddings1, embeddings2)
has always worked (no memory limits have ever come up). Just trying to think of ways to speed this up.
The first for-loop does not matter. I would keep it like this.
Otherwise:
Note, here is a method for pairwise cosine similarity score computation (not yet released with pip): https://github.com/UKPLab/sentence-transformers/blob/b5a85a826faeab9ad781eeba700308e7913c9700/sentence_transformers/util.py#L79
pytorch_cos_sim computes all combinations, pairwise_cos_sim only cossim(sent1[i], sent2[i])
Hello,
This snippet shows comparing
sentences1
vssentences2
throughutil.pytorch_cos_sim()
.Lets say I have two lists with 100 books each.
I could do this --
Any advice on a faster approach that avoids the first for-loop?