Have gpt4all Python library automatically use GPT4All desktop models, if installed

[simonw]

Feature request

Currently, if I pip install gpt4all from https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/python and then use it to run a model it appears to download a fresh copy, even if I already have the GPT4All desktop app installed.

It would be really neat if the Python library could detect this and reuse the existing model, rather than downloading a fresh copy.

Related:

• https://github.com/simonw/llm-gpt4all/issues/5

Motivation

I love the gpt4all Python library and I use it for my LLM tool - https://simonwillison.net/2023/Jul/12/llm/ - but I'd like it if existing GPT4All users got a better experience.

Your contribution

-

[simonw]

Relevant code: https://github.com/nomic-ai/gpt4all/blob/60627bd41f49c5363c7e2abecc9b3be6ce59570f/gpt4all-bindings/python/gpt4all/gpt4all.py#L16

[simonw]

It looks like the desktop app stores models here:

~/Library/Application Support/nomic.ai/GPT4All/

[cosmic-snow]

I guess the question here is whether a library (the bindings) should automatically go and find some .ini file somewhere in the current user's home folder and then change behaviour based on that.

Currently, the way to do it is to init with params model_name='...', model_path='...', allow_download=False. It's not very ergonomic when e.g. in a REPL, but it won't potentially cause confusion otherwise.

(Not that I'd have strong objections to this, though. I just fix my paths.)

[simonw]

Yeah a library that behaves differently if you have a desktop app installed is a little bit strange.

Maybe the library could take a list of search paths for models? Something like this:

gpt4all = GPT4All(..., model_search_paths=("~/Library/Application Support/nomic.ai/GPT4All/",))

Then when you load a model it checks folders in those paths first. It would still store models it downloads to its own directory though.

[jwoglom]

Just in case anyone else was wondering whether there's a relatively easy solution to avoid re-downloading... you can just symlink the files into the .cache/gpt4all folder:

$ for f in ~/Library/Application\ Support/nomic.ai/GPT4All/*.gguf; do ln -s "$f" "$HOME/.cache/gpt4all/"$(basename "$f"); done