search

intel-analytics / text-generation-webui

A Gradio Web UI for running local LLM on Intel GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) using IPEX-LLM.
GNU Affero General Public License v3.0
14 stars 8 forks source link

Add model.half() when model load #25

Closed hkvision closed 5 months ago

hkvision commented 5 months ago

@sgwhat To save some memory?

Related issue may: https://github.com/intel-analytics/ipex-llm/issues/10575 If on Arc750

sgwhat commented 5 months ago

sure.