Validation error for GPT4All when using Llama2

[oparoz]

1. I'm running Ubuntu and made sure there were no errors at install time
2. I downloaded exclusively the Llama2 model
3. I selected the Llama2 model in the admin section and all flags are green
4. Using the assistant, I asked for a summary of a text
5. A few minutes later, I get a notification that the process had failed
6. In the logs, I see this:

   Warning | llm | Invalid model file Traceback (most recent call last):   File "/nextcloud/apps/llm/src-py/index.py", line 23, in <module>     llm = GPT4All(model=dir_path + "/models/"+args.model, n_threads=int(threads))   File "/var/www/nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/load/serializable.py", line 74, in __init__     super().__init__(**kwargs)   File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All __root__   Unable to instantiate model (type=value_error)

From a user perspective, it looks like the wrong model is being used

[oparoz]

I installed the GPT4All model and get the same error, so not sure what is going on.

[marcelklehr]

Hi, I'm sorry for the delay. I'll look into this.

From a user perspective, it looks like the wrong model is being used

This is likely not the case. GPT4All is the library and gpt4all falcon is the model, they are separate.

[marcelklehr]

Which OS, which CPU architecture?

[oparoz]

Thank you for looking into this.

OS: Ubuntu 22.04.3 LTS Arch: amd64

I updated to the latest version and the error now looks different:

[llm] Warning:  /nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/__init__.py:40: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported.
  warnings.warn(
 /nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/__init__.py:40: UserWarning: Importing PromptTemplate from langchain root module is no longer supported.
  warnings.warn(
Invalid model file
Traceback (most recent call last):
  File " /nextcloud/apps/llm/src-py/index.py", line 23, in <module>
    llm = GPT4All(model=dir_path + "/models/"+args.model, n_threads=int(threads))
  File " /nextcloud/apps/llm/python/lib/python3.10/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File " /nextcloud/apps/llm/python/lib/python3.10/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)

[luxuskommunismus]

I encounter the same error. I am running Nextcloud in a Docker container on a VPS. I suspect it has to do with my VPS provider not passing AVX-CPU-flags to the server. These are required for the GPT4ALL ecosystem though

[oparoz]

AVX-CPU-flags [...] These are required for the GPT4ALL ecosystem though

This is not in the README. Is this a requirement for using these LLMs? That could explain my problem as the Celeron in this box doesn't support those instructions.

[luxuskommunismus]

Yeah, it's missing in the README of this app, but its a requirement of the underlying architecture, see here: https://github.com/nomic-ai/gpt4all

If this is indeed the cause of the problem, it should probably be in the README of this app as well.

[oparoz]

Thank you so much for highlighting this. I won't try again until I have migrated to a more powerful CPU :)

[Forza-tng]

I have a Ryzen CPU, which does have AVX2 support. I am also getting a similar error, no matter which of the three models I choose.

{
    "reqId": "h5ZekSOB3CR3x587ry4n",
    "level": 2,
    "time": "2023-11-11T22:58:05+00:00",
    "remoteAddr": "",
    "user": "--",
    "app": "llm",
    "method": "",
    "url": "--",
    "message": "/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported.\n  warnings.warn(\n/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing PromptTemplate from langchain root module is no longer supported.\n  warnings.warn(\nTraceback (most recent call last):\n  File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/src-py/index.py\", line 23, in <module>\n    llm = GPT4All(model=dir_path + \"/models/\"+args.model, n_threads=int(threads))\n          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/load/serializable.py\", line 97, in __init__\n    super().__init__(**kwargs)\n  File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/pydantic/v1/main.py\", line 341, in __init__\n    raise validation_error\npydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All\n__root__\n  Failed to retrieve model (type=value_error)\n",
    "userAgent": "--",
    "version": "27.1.3.2",
    "data": { "app": "llm" }
}

/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported.
warnings.warn(
/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/__init__.py:40: UserWarning: Importing PromptTemplate from langchain root module is no longer supported.
warnings.warn(
Traceback (most recent call last):
File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/src-py/index.py\", line 23, in <module>
llm = GPT4All(model=dir_path + \"/models/\"+args.model, n_threads=int(threads))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/langchain/load/serializable.py\", line 97, in __init__
super().__init__(**kwargs)
File \"/var/www/domains/tnonline.net/htdocs/nextcloud/apps/llm/python/lib/python3.11/site-packages/pydantic/v1/main.py\", line 341, in __init__
raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
Failed to retrieve model (type=value_error)

The downloaded models have the following sha256:

259a403e5c4f098936377a62b8d6f05d3c99bb9aebe36166c13bf3cdd5bcf20b  gpt4all-falcon-q4_0.gguf
e4dcdde73bd4ea6975a588c0ace974734cd7419d68ba5d58efa12b37f7eb802f  leo-hessianai-13b-chat-bilingual.Q4_K_M.gguf
08a5566d61d7cb6b420c3e4387a39e0078e1f2fe5f055f3a03887385304d4bfa  llama-2-7b-chat.Q4_K_M.gguf

[dpi027]

I had the same issue and after strace-ing the cron.php job after using the assistant, I found out that the code was looking for a .bin file, not the .gguf file in the apps/llm/src-py/models folder.

So I downloaded the file ggml-model-gpt4all-falcon-q4_0.bin, and renamed it to gpt4all-falcon-q4_0.gguf.bin to match the name of the model (but with a .bin extension added). It worked.

I don't know if the format between the two files are different or if it is just a naming issue.

[luxuskommunismus]

These are two different errors though, difference is in the last line. The first is: "Unable to instantiate model (type=value_error)", which in this case I think has to do with missing AVX-support. The second says "Failed to retrieve model (type=value_error)", also reported here #39

[marcelklehr]

Yes, #39 should be fixed with v1.2.1 now

[marcelklehr]

I've added the AVX requirement to the README now