Open clemarjonlaguardia opened 1 month ago
How can I set the weight_dtype as GGFU? Because it doesn't appeared in the list.
No need to set, it will be used automatically, because only some of the layers use Q4 quantization, and the rest of the layers use the data type in weight_dtype
How can I set the weight_dtype as GGFU? Because it doesn't appeared in the list.