search

huggingface / text-generation-inference

Large Language Model Text Generation Inference
http://hf.co/docs/text-generation-inference
Apache License 2.0
8.48k stars 968 forks source link

Feature request: Classifier-Free Guidance (CFG) Sampling #554

Closed dongs0104 closed 2 months ago

dongs0104 commented 1 year ago

Feature request

Stay on topic with Classifier-Free Guidance CFG brings non trivial improvements on many standard benchmarks.

Motivation

The response quality of LLMs using CFG averaged similarly to standard models twice the size. For example, LLaMA-7B with CFG outperformed LLaMA-13B on the Lambada text-generation benchmark. This is at the expense of inference time.

authors contribute on transformers https://github.com/huggingface/transformers/pull/24654

Your contribution

I'm not ready to contribution on TGI. this is a backlog.

github-actions[bot] commented 2 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.