Question about model config. - Githubissues

InternLM / InternLM-XComposer

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

2.14k stars 133 forks source link

Question about model config. #178

Closed liuxianyi closed 4 months ago

liuxianyi commented 4 months ago

    gen_config = dict(
        num_beams=5,
        do_sample=False,
        min_length=1,
        repetition_penalty=1.5,
        length_penalty=1.0,
        temperature=1.0,
        max_new_tokens=500,
    )

what is the num_beams and do_sample? when composed, how will they take effect?

yhcao6 commented 4 months ago

num_beams is related to beam search and determines how many candidate sequences are considered at each step of the generation process. A higher value can lead to better quality outputs but requires more computational resources.
do_sample decides whether the generation process should randomly sample from the probability distribution of next tokens, or just choose the highest probability next token at each step.

You can refer to https://huggingface.co/docs/transformers/generation_strategies for more details.