search

huggingface / pytorch-openai-transformer-lm

🐥A PyTorch implementation of OpenAI's finetuned transformer language model with a script to import the weights pre-trained by OpenAI
MIT License
1.51k stars 285 forks source link

Simplifying the use of the model to perform different tasks #25

Closed rodgzilla closed 6 years ago

rodgzilla commented 6 years ago

This patch contains the code discussed about in #24.

Its goal is to simplify the allow users to use this model for many different tasks, as presented in the research paper. For example, let's say you want to finetune the network to classify texts, you just have to create a DoubleHeadModel with a classification head and use the ClassificationLossCompute class.

The SimilarityHead has not been tested yet and the SimilarityLossCompute is missing as I don't know how this kind of task works.