search

albertkx / Berkeley-Crossword-Solver

ACL 2022
MIT License
122 stars 20 forks source link

Alternatives for hard negatives #8

Open logachevpa opened 1 year ago

logachevpa commented 1 year ago

Have you tried using bi encoder self for hard negative mining? Like second stage of training QA model, after using tfidf negatives, or from the beginning (reducing source dependencies). Maybe it could converge into a better model. Or maybe it would be worse due to overfitting.

Thank you for the work and publishing the source code!

Eric-Wallace commented 1 year ago

No we havent tried that but its a great idea!