search

huggingface / blog

Public repo for HF blog posts
https://hf.co/blog
2.36k stars 745 forks source link

Potential Dead Links In Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models #64

Closed ethen8181 closed 3 years ago

ethen8181 commented 3 years ago

Hi, @patrickvonplaten, in the Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models blog post. https://huggingface.co/blog/warm-starting-encoder-decoder

Some of the links to the additional notebooks seem to redirect me to the same master blog post. Is this expected?

for BERT2BERT on CNN/Dailymail (a condensed version of this notebook), click here.
for RoBERTaShare on BBC XSum, click here.
for BERT2Rnd on WMT14 En to→ De, click here.
for RoBERTa2GPT2 on DiscoFuse, click here.
patrickvonplaten commented 3 years ago

Hey @ethen8181 - I didn't manage to finish those "shorter" notebooks yet. I'll disable the links for now - thanks for the issue!

rcalix1 commented 1 year ago

Hello Patrick,

I am looking for all examples of warm starting Transformers with the Bert2gpt configuration. Has Huggingface developed the code yet to warm start Llama/Alpaca on the decoder? Can you please let me know, or point me in the direction closest to this objective.

Thanks, Ricardo