They apply whitening to SimCSE to mitigate anisotropy.
2. What is amazing compared to previous works?
Their method improved the performance of STS and transfer tasks while eliminating anisotropy.
3. Where is the key to technologies and techniques?
shuffle group whitening: Splitting the d dimension into k random groups and whitening in each group, which can be done multiple times, also acts as data expansion.
4. How did evaluate it?
Their method (WhitenCSE) constantly achieves high performance at STS tasks and transfer tasks.
5. Is there a discussion?
Their method mitigate anisotropy (alignment and uniformity).
a1da4
commented
4 months ago
0. Paper
1. What is it?
They apply whitening to SimCSE to mitigate anisotropy.
2. What is amazing compared to previous works?
Their method improved the performance of STS and transfer tasks while eliminating anisotropy.
3. Where is the key to technologies and techniques?
4. How did evaluate it?
5. Is there a discussion?
Their method mitigate anisotropy (alignment and uniformity).
6. Which paper should read next?