-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTor…
-
### 🚀 The feature, motivation and pitch
Huggingface generation interface allows beam search with nonzero temperature, allowing to generate more diverse beam search. The current version with zero temp…
-
I have been going through `generate/synthesis_exp.py` script. I noticed that default num_beams is set to 1 for the generation. Does this mean you're doing greedy decoding for "Beam-Search" entry of th…
-
Env: torch2.4 cuda 12.4 unsloth main
below is the code errored
```
from unsloth import FastLanguageModel
import torch
model_id="unsloth/gemma-2-2b-it-bnb-4bit"
model, tokenizer = FastLanguageM…
-
Dears,
I failed to run Llama-2-7b-chat-hf on NPU, please give me a hand.
1. I converted the mode by below command, and got two models,
a) optimum-cli export openvino --task text-generation -m Meta-…
-
Could you add a beam search parameter for high level API generation? Thanks.
-
### Motivation
Does sglang now support beam search? I tried using the 'best_of' parameter, but the output seems to be the same as if it wasn't used.
### Related resources
_No response_
-
Hello Exllama friends,
I was curious what the thoughts are on implementing beam search in v2.
In the v1. Beam search was implemented in the core generator.
I was curious what would the requirem…
-
Hello, if I want to view multiple descriptions generated by the final beam search strategy for the same image, in which code should I output this?
-
Just to make sure that issue won't be missed, will duplicate [my response](https://github.com/OpenNMT/CTranslate2/issues/1493#issuecomment-2220299041) here:
I faced the same issue. However, when re…