search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
133.17k stars 26.59k forks source link

implementation of transformers for abstractive summarization task #4562

Closed fatihbeyhan closed 4 years ago

fatihbeyhan commented 4 years ago

Hello, I am new to whole NLP world and PyTorch. I am trying to learn the concepts and that is taking some time for a rookie. I have a project to finish and I want to implement transformers & BERT on my abstractive summarization project. I tried to find some implementation tutorial on this topic but I could not find any. Do you guys have any suggestions about clear implementation of any pre-trained model that I can fine-tune on my dataset to get some solid results. I am not looking for this just for finishing up the project but also learn how to implement. Therefore, I need a clear tutorial.

Data: I am using 0.25 of XSum dataset so I have 45k news and their one-sentence summary.

Thank you in advance.

patrickvonplaten commented 4 years ago

@sshleifer might be able to help you here. @sshleifer - hope it's fine that I link you here :-)

patrickvonplaten commented 4 years ago

This might also help: https://github.com/huggingface/transformers/pull/4539/files?short_path=3a2ba7b#diff-3a2ba7b492f00029d14cec3994b73ac7

fatihbeyhan commented 4 years ago

This might also help: https://github.com/huggingface/transformers/pull/4539/files?short_path=3a2ba7b#diff-3a2ba7b492f00029d14cec3994b73ac7

Thank you very much. I will look and try to implement and let you know about the result!

fatihbeyhan commented 4 years ago

This might also help: https://github.com/huggingface/transformers/pull/4539/files?short_path=3a2ba7b#diff-3a2ba7b492f00029d14cec3994b73ac7

it seems working! thank you!