-
I have been going through `generate/synthesis_exp.py` script. I noticed that default num_beams is set to 1 for the generation. Does this mean you're doing greedy decoding for "Beam-Search" entry of th…
-
Just to make sure that issue won't be missed, will duplicate [my response](https://github.com/OpenNMT/CTranslate2/issues/1493#issuecomment-2220299041) here:
I faced the same issue. However, when re…
-
Could you add a beam search parameter for high level API generation? Thanks.
-
Hi,
I am getting around 3% wer in fast-beam-search and greedy-search. However, I am getting 70% WER when I use fast-beam-search-ngram. My decode configuration looks as below. I am using pruned_tran…
-
Hello Exllama friends,
I was curious what the thoughts are on implementing beam search in v2.
In the v1. Beam search was implemented in the core generator.
I was curious what would the requirem…
-
Beam search is generally not supported by our models, though the flag exists. It appears to be supported in `lstm`; it is unclear to me whether it's supported by `feature_invariant_transformer`, `tran…
-
`
model_name = "meta-llama/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = Model()
model.init(model_name, use_quant=True, weight_dtype="in…
-
This feature was proposed by @spion in https://github.com/ggerganov/llama.cpp/issues/2813#issuecomment-1694390583
> In some cases, its useful to do constrained evaluation of logits based on a union…
-
investigate beam search to do a greedy search of the HS Code tree.
-
I'm using the scorer generator provided `generate_scorer_package`. I'm also using (e.g., SentencePiece) to build a unigram language model, where the decoder predicts the size of the language model. Ho…