search

intel-analytics / text-generation-webui

A Gradio Web UI for running local LLM on Intel GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) using IPEX-LLM.
GNU Affero General Public License v3.0
14 stars 8 forks source link

Add warmup when loading models #15

Closed Zhangky11 closed 6 months ago

Zhangky11 commented 6 months ago

Description

Add warmup steps in modules/ui_model_menu.py. Performance in https://github.com/analytics-zoo/nano/issues/1046

Zhangky11 commented 6 months ago

Is this about directly inputting a "hello" to chat for a warm-up?

That's right. And the "hello" input and its reply won't be recorded in history chat. It won't influence future chat.

sgwhat commented 6 months ago

Is this about directly inputting a "hello" to chat for a warm-up?

That's right. And the "hello" input and its reply won't be recorded in history chat. It won't influence future chat.

Nice!