Fine-tuning for paraphrasing tasks

[anmoljagetia]

❓ Questions & Help

I asked on SO and was downvoted since it is considered "off-site" and is against their Terms of Service. My question is somewhat simple. How do I fine-tune a GPT-2 model for the task of paraphrasing like the paper: https://www.aclweb.org/anthology/D19-5623.pdf

A link to my SO question : https://stackoverflow.com/questions/61115488/how-to-fine-tune-gpt-2-for-paraphrasing?noredirect=1#comment108120354_61115488

Details

My question is somewhat simple. How do I fine-tune a GPT-2 model for the task of paraphrasing like the paper: https://www.aclweb.org/anthology/D19-5623.pdf

Is there a way to achieve this with huggingface-transformers ?

[janniks]

This might help: https://huggingface.co/transformers/usage.html#sequence-classification

[anmoljagetia]

^^ These are inference examples. Do you know how can I retrain ?

[kormilitzin]

I would stress that this topic is quite interesting and useful. A good generative model for paraphrasing may help with text classification with small datasets. Backtranslation (for example) has shown as an effective way to augment the training data and boost performance of a classifier. However, echoing the @anmoljagetia, fine-tuning on the target domain may also bee important.

[beyhangl]

@anmoljagetia did you find any method to retrain the model to generate paraphrase sentence?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.