huggingface / pytorch-openai-transformer-lm

🐥A PyTorch implementation of OpenAI's finetuned transformer language model with a script to import the weights pre-trained by OpenAI
MIT License
1.51k stars 285 forks source link

Implementation of Similarity Head #60

Open TEGELB opened 3 years ago

TEGELB commented 3 years ago

Similarity Head and Loss function were tested on the STS-B dataset, achieving nearly the same performance as reported (82.45% PC relative to the 82% in the paper). I can provide the necessary changed code for loading the dataset and reproducing my results if wanted.