source 문장을 target style의 문장으로 바꾸는 Generative Style Transformer (GST) 를 제안하였습니다.
기존의 Transformer 와 동일하게, 대량의 unsupervised language modeling pretraining 을 진행한 이후에 parallel style corpora 를 이용해서 style-transfer 테스크를 fine-tuning 을 하여 GST를 학습하였습니다.
Novelty로서 source 문장에서 style attribute 를 삭제하는 새로운 method 를 제안하였습니다 (`Delete Retrieve Generate')
기존의 BLEU 보다 사람의 evaluation 을 더 잘 반영하는 GLUE metric 을 제안하였습니다.
5 datasets on sentiment, gender and political slant 테스크에서 SOTA를 달성하였습니다.
Abstract (요약) 🕵🏻♂️
Text style transfer is the task of transferring the style of text having certain stylistic attributes, while preserving non-stylistic or content information. In this work we introduce the Generative Style Transformer (GST) - a new approach to rewriting sentences to a target style in the absence of parallel style corpora. GST leverages the power of both, large unsupervised pre-trained language models as well as the Transformer. GST is a part of a larger `Delete Retrieve Generate' framework, in which we also propose a novel method of deleting style attributes from the source sentence by exploiting the inner workings of the Transformer. Our models outperform state-of-art systems across 5 datasets on sentiment, gender and political slant transfer. We also propose the use of the GLEU metric as an automatic metric of evaluation of style transfer, which we found to compare better with human ratings than the predominantly used BLEU score.
이 논문을 읽어서 무엇을 배울 수 있는지 알려주세요! 🤔
Generative Model 이기 때문에, GPT와 유사한 성격인지 아니면 BERT 와 유사한 성격인지 확인해 봐야 합니다.
최근에 간단한 self-supervised 기법을 활용해서 성능을 높이는 방식들이 사용되고 있는데, 본 논문에서 제시한 delete 가 어떤 novelty 가 있고 이런 간단한 기법이 accept 되기 위해서 저자가 제시한 논리 전개를 파악할 수 있습니다.
최근에 human evaluation 과 correlation 이 높은 metric 들에 대한 연구가 이어지고 있는데, 기존의 연구들과 어떤 차이점이 있는지 확인해 볼 수 있습니다.
어떤 내용의 논문인가요? 👋
Abstract (요약) 🕵🏻♂️
Text style transfer is the task of transferring the style of text having certain stylistic attributes, while preserving non-stylistic or content information. In this work we introduce the Generative Style Transformer (GST) - a new approach to rewriting sentences to a target style in the absence of parallel style corpora. GST leverages the power of both, large unsupervised pre-trained language models as well as the Transformer. GST is a part of a larger `Delete Retrieve Generate' framework, in which we also propose a novel method of deleting style attributes from the source sentence by exploiting the inner workings of the Transformer. Our models outperform state-of-art systems across 5 datasets on sentiment, gender and political slant transfer. We also propose the use of the GLEU metric as an automatic metric of evaluation of style transfer, which we found to compare better with human ratings than the predominantly used BLEU score.
이 논문을 읽어서 무엇을 배울 수 있는지 알려주세요! 🤔
레퍼런스의 URL을 알려주세요! 🔗
https://arxiv.org/abs/1908.09368