Open danerlt opened 3 months ago
@dorianbrown Can you help me review the code, please? Thank you.
@danerlt Sure thing, I'll have time to take a look sometime in the next week. Looking forward to it 👍
I think this is a problem that needs addressing, and from what I can tell there are two ways to go (see this PR for some of the thoughts I just shared), so either fix the epsilon approach and keep the IDF as is, or remove that negativity checking, and go for this "smoothed" IDF function.
I'm not working on this topic as intensively as before, but do you know if there's a reason to prefer one of the two from a ranking perspective? Computationally the changed IDF function seems preferable.