search

UppuluriKalyani / ML-Nexus

ML Nexus is an open-source collection of machine learning projects, covering topics like neural networks, computer vision, and NLP. Whether you're a beginner or expert, contribute, collaborate, and grow together in the world of AI. Join us to shape the future of machine learning!
https://discord.gg/n2D4RqnU
MIT License
40 stars 69 forks source link

Feature request: Text Summarization #339

Open rishikaa1 opened 2 days ago

rishikaa1 commented 2 days ago

Is your feature request related to a problem? Please describe. Yes, summarizing English text efficiently and accurately is a challenging task, especially for users who need to quickly extract key information from large documents. Existing solutions may not always produce concise and relevant summaries, leading to time inefficiency and potential misunderstanding of important content.

Describe the solution you'd like I propose implementing an English Text Summarization model using the mBART model from Hugging Face's Transformers library and the XSum dataset. This feature would allow users to input English text and receive concise, accurate summaries in English.

Describe alternatives you've considered While there are several summarization tools available, many lack the accuracy and fluency we aim to achieve. Alternatives include using simpler extractive summarization techniques or fine-tuning other language models. However, utilizing mBART with the XSum dataset offers a promising balance of performance and output quality for English summarization.

Approach to be followed (optional)

  1. Utilize the XSum dataset from Hugging Face for English summarization.
  2. Preprocess the XSum dataset and use the mBART model for fine-tuning.
  3. Train the model on English summarization tasks and evaluate its performance using metrics such as ROUGE for summarization quality.
  4. Implement a user-friendly interface for inputting English text and displaying summaries.
  5. Conduct thorough testing with various types of English texts to ensure consistency and quality.
  6. Provide comprehensive documentation for users and developers.

Additional context The XSum dataset contains primarily English data, so our model will focus exclusively on English-to-English summarization, as opposed to the multilingual text summarization proposed earlier.

github-actions[bot] commented 2 days ago

Thanks for creating the issue in ML-Nexus!🎉 Before you start working on your PR, please make sure to: