search

Shivanandroy / simpleT5

simpleT5 is built on top of PyTorch-lightning⚡️ and Transformers🤗 that lets you quickly train your T5 models.
MIT License
387 stars 62 forks source link

simpleT5 for Grammatical Error Correction #15

Closed pradeepdev-1995 closed 3 years ago

pradeepdev-1995 commented 3 years ago

How to use simpleT5 for Grammatical Error Correction

Shivanandroy commented 3 years ago

Hi @pradeepdev-1995 You can use simpleT5 for grammar error correction or any other sequence to sequence tasks. You can label your grammatically incorrect sentences as source_text and correct sentences as target_text and use any of the t5/mT5/byT5 models for fine tuning.

pradeepdev-1995 commented 3 years ago

@Shivanandroy Thank you. Shall I use the specific GEC model - flexudy/t5-small-wav2vec2-grammar-fixer for fine-tuning using simpleT5?

Shivanandroy commented 3 years ago

You can use any T5 model of your choice. My recommendation will be to use you a pretrained t5/mt5 model and then fine tune it on your corpus.

pradeepdev-1995 commented 3 years ago

@Shivanandroy okay

So If I use simpleT5 for GEC then what prefix should I put like summarize in summarization

# T5 model expects a task related prefix: since it is a summarization task, we will add a prefix "summarize: "
df['source_text'] = "summarize: " + df['source_text']
Shivanandroy commented 3 years ago

refer to #2