search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
135.02k stars 27.02k forks source link

Task specific params for pegasus-large to allow finetuning with correct generation_parameters #6774

Closed sshleifer closed 4 years ago

sshleifer commented 4 years ago 
'task_specific_params': {'summ_xsum': {'max_length': 56, 'length_penalty': 0.8}}}
sshleifer commented 4 years ago

Done!

# Config values that vary between checkpoints: for testing and conversion
task_specific_params = {
    # These are task specific params for pegasus-large and normal params for finetuned checkpoints
    "summarization_xsum": {"length_penalty": 0.8, "max_length": 64, "max_position_embeddings": 512},
    "summarization_cnn_dailymail": {"length_penalty": 0.8, "max_length": 128, "max_position_embeddings": 1024},
    "summarization_newsroom": {"length_penalty": 0.8, "max_length": 128, "max_position_embeddings": 512},
    "summarization_wikihow": {"length_penalty": 0.6, "max_length": 256, "max_position_embeddings": 512},
    "summarization_multi_news": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_reddit_tifu": {"length_penalty": 0.6, "max_length": 128, "max_position_embeddings": 512},
    "summarization_big_patent": {"length_penalty": 0.7, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_arxiv": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_pubmed": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_gigaword": {"length_penalty": 0.6, "max_length": 32, "max_position_embeddings": 128},
    "summarization_aeslc": {"length_penalty": 0.6, "max_length": 32, "max_position_embeddings": 512},
    "summarization_billsum": {"length_penalty": 0.6, "max_length": 256, "max_position_embeddings": 1024},
    # this last entry is useless -- just for consistency
    "summarization_large": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
}