Open flatsiedatsie opened 4 months ago
In the current list of default models there is an odd ommission. While most models also have a 32 Bit variant available, Mistral 7B does not.
The practical result is that Linux users are missing out on performance, since their implementation of WegGPU currently only supports 32 Bit.
E.g.
Would it be possible to make sure all base models always have a 32 bit option available until Linux also supports 16 bit WebGPU?
Also upgrading Mistral to the V3 version with 32K context and function calling support would still be wonderful.
In the current list of default models there is an odd ommission. While most models also have a 32 Bit variant available, Mistral 7B does not.
The practical result is that Linux users are missing out on performance, since their implementation of WegGPU currently only supports 32 Bit.
E.g.
Would it be possible to make sure all base models always have a 32 bit option available until Linux also supports 16 bit WebGPU?