We view Large Language Models as stochastic language layers in a network, where the learnable parameters are the natural language prompts at each layer. We stack two such layers, feeding the output of one layer to the next. We call the stacked architecture a Deep Language Network - DLN
Generate a random seed for every request if the LLM was initialized with a seed.
Seeds provided when LLM.generate(prompt, seed) is called has priority over the internal seed.
Seed per request support varies by model. As of today, only gpt-3.5-turbo-instruct supports it. gpt-4-turbo supports it to some extension. Other models such as gpt-3, gpt-4, and VLLM models (phi-2, llama2, etc) ignore the seed.
MarcCote
commented
4 months ago
Generate a random seed for every request if the LLM was initialized with a seed. Seeds provided when
LLM.generate(prompt, seed)is called has priority over the internal seed.Seed per request support varies by model. As of today, only
gpt-3.5-turbo-instructsupports it.gpt-4-turbosupports it to some extension. Other models such as gpt-3, gpt-4, and VLLM models (phi-2, llama2, etc) ignore the seed.