nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
https://nomic.ai/gpt4all
MIT License
70.65k stars 7.7k forks source link

Searching for models returns f32 models and no option to select the quant size #2526

Open sammcj opened 4 months ago

sammcj commented 4 months ago

Bug Report

Searching for models gives no option to select the quant size and seems to favour f32 which is a bit overkill IMO 🤣, searching with a specific quant size returns a completely different model

image image

Steps to Reproduce

  1. Search for Phi 3.1
  2. Returns results for F32
  3. No option to select the quant type/size
  4. Search for Phi 3.1 Q6_K
  5. Note that it returns results for llama3?

Expected Behaviour

Be able to search for a model and have it suggest a reasonable quant size (e.g. Q6_K, Q5_K_M and Q4_K_M) by default.

Be able to search for a model and it's quant size and have it return that model and quant size (rather than a different model entirely)

Your Environment

uninstall-your-browser commented 4 months ago

It would be nicer if it just listed the quant options available like lm-studio does