LCPP Default is set to 4, which is a bit too much in my opinion. Setting to 2 saves VRAM (0.5-1%?), some compute and some electricity if set to 2, at the expense of some potential performance (prompt processing?), that I do not notice in usage. 2 is thus my own setting.
LCPP Default is set to 4, which is a bit too much in my opinion. Setting to 2 saves VRAM (0.5-1%?), some compute and some electricity if set to 2, at the expense of some potential performance (prompt processing?), that I do not notice in usage. 2 is thus my own setting.
https://github.com/ggerganov/llama.cpp/pull/6017