search

AkihikoWatanabe / paper_notes

たまに追加される論文メモ
https://AkihikoWatanabe.github.io/paper_notes
17 stars 0 forks source link

Assisted Generation: a new direction toward low-latency text generation, 2023 #675

Open AkihikoWatanabe opened 1 year ago

AkihikoWatanabe commented 1 year ago

https://huggingface.co/blog/assisted-generation

AkihikoWatanabe commented 1 year ago

1 line加えるとtransformerのgenerationが最大3倍程度高速化されるようになったらしい

AkihikoWatanabe commented 1 year ago

image

assistant modelをロードしgenerateに引数として渡すだけ image