Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

[jinmang2]

집현전 중급반 스터디

• 2022년 5월 29일 일요일 9시
• 허은진님 원혜진님 정민지님 최석민님 발표
• 논문 링크: https://arxiv.org/abs/1908.10084

Abstract

BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT. The construction of BERT makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.

[jinmang2]

• 발표링크: https://www.youtube.com/watch?v=Oy3Jv9P7i5w&list=PL2tRglRS_GqjCcBVM26gYNdoOnX12lLIh&index=10

[HanNayeoniee]

@jinmang2 올려주신 주소가 XLNet 논문 발표링크 같습니다.

• Sentence-BERT 발표 링크: https://www.youtube.com/watch?v=fj1trYPTMA8&list=PL2tRglRS_GqjCcBVM26gYNdoOnX12lLIh&index=10

[jinmang2]

@HanNayeoniee 감사합니다 나연님! :-)