search

huggingface / huggingface-llama-recipes

531 stars 59 forks source link

[Add] Assisted Decoding with Llama 1B and 3B models #35

Closed ariG23498 closed 1 month ago

ariG23498 commented 1 month ago

This PR adds the following:

  1. Assisted decoding with Llama 3.1 8B (base) + Llama 3.2 1B (assistant)
  2. Assisted decoding with Llama 3.1 70B (base) + Llama 3.2 3B (assistant)

small big

CC: @osanseviero @Vaibhavs10 @gante