Open JamieVC opened 3 months ago
The default values are determined by the model configuration. For instance, if you use the Llama-2-chat-hf model, the default settings are temperature=0.9 and top_p=0.6. You can modify these settings in the generation_config.json
file located in the model folder.
What are default values of max_generated_tokens, top_k, top_p, and temperature? If user doesn't set all parameters in
generate_kwargs
such as the example below, it should use default values. How do we get them in which source file?refer to: https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/example/GPU/LlamaIndex/rag.py