Closed ThiloteE closed 1 month ago
I have added this model at the location "order": "z",
, because I fear there might be merge conflicts with #2750
VAGOsolutions confirm its RAG capabilities in their German RAG benchmark:
I've downloaded it and checked some fields, and they're all fine: md5sum, name, filename, filesize, quant, type, parameters
I have not looked at their site/blog to verify the templates, however a quick test with them went well.
Adds a models3.json entry for Qwen2-1.5B-Instruct
Description of Model
It is a tiny bilingual model and at the date of writing with very strong results in benchmarks (for its parameter size). It supports a context of up to 32768. Because of its model size it has very fast responses, even when doing inference on CPU. This LLM is LITERALLY for all. Since the model fits into 4GB of RAM (just barely, if the Operating System and other apps also need RAM) or alternatively into 3GB of VRAM, this will be the workhorse of the desperate and hardware poor.
Personal Impression:
I got the impression the model is very task focused and this is the reason, why I chose
Below is an instruction that describes a task. Write a response that appropriately completes the request.
as system prompt. Since the model is relatively small, its responses may seem not very coherent or intelligent, but it works surprisingly well with GPT4All's LocalDocs feature. It is like the model was made for RAG. Its long context adds to that. It mainly will appeal to English and Chinese speaking users.Checklist before requesting a review