search

nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
https://nomic.ai/gpt4all
MIT License
70.96k stars 7.72k forks source link

[Feature] Crash: Support old MPT GGUF conversions with duplicated output tensor #2329

Open dlippold opened 6 months ago

dlippold commented 6 months ago

Bug Report

The fine-tuned MPT model from https://huggingface.co/maddes8cht/mosaicml-mpt-7b-instruct-gguf/ in quantization Q4_1 was usabel in release 2.7.2 but not longer in 2.7.3 and later. In particular it is currently not usable.

When I try to load the model file I get the following error message:

Could not load model due to invalid model file for mosaicml-mpt-7b-instruct-Q4_1.gguf

The reason of the problem may have to do with #2006

Steps to Reproduce

  1. Download the model file from the specified URL
  2. Start GPT4all
  3. Choose the downloaded model file

Expected Behavior

The model file should be loaded.

Your Environment

cebtenzzre commented 6 months ago

I fixed this upstream in https://github.com/ggerganov/llama.cpp/pull/6139 which should make it into the next release of GPT4All (already included in #2310).

dlippold commented 5 months ago

Version 2.8.0 crashes when loading the model named above.