search

Shivanandroy / Blog

1 stars 0 forks source link

Fine Tuning T5 Transformer Model with PyTorch - Shivanand Roy | Deep Learning #1

Open utterances-bot opened 3 years ago

utterances-bot commented 3 years ago

Fine Tuning T5 Transformer Model with PyTorch - Shivanand Roy | Deep Learning

http://shivanandroy.com/fine-tune-t5-transformer-with-pytorch/

divyadixit commented 3 years ago

Hello Shivanand,

I came across your notebook on Fine Tuning T5 model using Pytorch and it's a very well documented approach you have put up which I had been looking for. I have been working on this summarization problem for a thesis project where I had been creating a datatset for Indian news articles and chanced upon your dataset. I would request your permission to use this dataset in our thesis for abstractive summarization. If yes, how can attribute it to you?

If you'd like to speak in detail and scope, I'll be happy to explain in detail.

Shivanandroy commented 3 years ago

Hi Divya - Thanks for your words. I'm glad I was able to help you.

The dataset used here is a part of Kaggle dataset You can attribute this in your thesis project.

Further, you can also have a look at simpleT5 for faster prototyping on your Indian news articles. Also, I would also encourage you to take the code from https://github.com/Shivanandroy/simpleT5/blob/main/simplet5/simplet5.py which is more stable, mature, and modular as it is built on top of PyTorch-lightning⚡️ and Transformers🤗

divyadixit commented 3 years ago

Thanks a lot Shivanand! I'm new to tranformer architecture and fine-tuning a transformer so we really appreciate your help :)

arjunKumbakkara commented 3 years ago

Hi Shivanandroy,

I was trying to use SimpleT5 and it worked , however when used for quantizing and converting to ONNX model using fastT5(internally of course).I am hit with an issue that only transformers 4.6.1 has the support of a particular method but simpleT5 needs minimum version of 4.8.2.

I tried with SimpleT5 0.0.7 which solved the issue, however, there are still errors. I am able to get the summary .But want to make sure this wouldnt cause an incompatibility issue in the future.

Thanks in advance

RosesRfree commented 2 years ago

After trainging t5 why doesnt one need to save the model and can jump to testing the summarization. How does it become portable?

jashokkumar83 commented 2 years ago

Dear Shivanand Roy,

I appreciate your hard work and tutorial on t5. Can you suggest me the calculation of accuracy, precision, recall, and f1 in this code?