search

allenai / PRIMER

The official code for PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization
Apache License 2.0
153 stars 32 forks source link

Would appreciate some guidance on using the model for MDS #30

Open yungsinatra0 opened 1 year ago

yungsinatra0 commented 1 year ago

Hi everyone, I've been working on a Colab notebook for a project that uses PRIMERA to summarize a given set of reports.

I've implemented it using the 'HF version' initialization taken from the HuggingFace page:

` from transformers import (

AutoTokenizer,

LEDConfig,

LEDForConditionalGeneration,

)

tokenizer = AutoTokenizer.from_pretrained('allenai/PRIMERA-multinews')

config=LEDConfig.from_pretrained('allenai/PRIMERA-multinews')

model = LEDForConditionalGeneration.from_pretrained('allenai/PRIMERA-multinews') `

Additionally, I've used code blocks from the example notebook (especially the document processing part and generating summary).

However, I'm not sure if I'm doing everything correctly as the summaries that I'm getting are a bit weird (they seem to focus on one document and only add small bits from other documents) + I don't really understand many parts of the code and how I can change it for my specific task.

Is there anyone who's successfully used & implemented this model that could help me out? I'd really appreciate some help. Thanks!