GPT4All CLI produces AttributeError: 'GPT4All' object has no attribute 'chat_completion'

[compilebunny]

System Info

This is on Ubuntu 22.

The instructions shown here https://docs.gpt4all.io/gpt4all_cli.html suggest that the CLI prerequisites be installed as follows:

python3 -m pip install --user --upgrade gpt4all typer

A second run of the above produces Requirement already satisfied: gpt4all in ./.local/lib/python3.10/site-packages (1.0.1) Requirement already satisfied: typer in ./.local/lib/python3.10/site-packages (0.9.0) Requirement already satisfied: tqdm in ./.local/lib/python3.10/site-packages (from gpt4all) (4.65.0) Requirement already satisfied: requests in ./.local/lib/python3.10/site-packages (from gpt4all) (2.28.2) Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.local/lib/python3.10/site-packages (from typer) (4.5.0) Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/lib/python3/dist-packages (from typer) (8.0.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->gpt4all) (2020.6.20) Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.local/lib/python3.10/site-packages (from requests->gpt4all) (1.26.15) Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.10/site-packages (from requests->gpt4all) (3.1.0) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->gpt4all) (3.3)

We then run: python3 app.py repl --model /model/path/ggml-gpt4all-l13b-snoozy.bin

Unfortunately, the first prompt provided to any model produces the error "AttributeError: 'GPT4All' object has no attribute 'chat_completion'".

Was the chat_completion method removed from gpt4all?

Should app.py be updated with a new method?

Information

• [X] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] backend
• [X] bindings
• [X] python-bindings
• [ ] chat-ui
• [ ] models
• [ ] circleci
• [ ] docker
• [ ] api

Reproduction

1. python3 -m pip install --user --upgrade gpt4all typer
2. python3 app.py repl --model /model/path/ggml-gpt4all-l13b-snoozy.bin
3. Enter any prompt

Expected behavior

Expected behavior: a response to the prompt from the model.

[mweckbecker]

How about you take a minute and look at the commit history? If you had done that you would've seen that the internal API is being reworked currently. You can simply adapt the basic prompt they ship. I've just done that for you now:

diff --git a/gpt4all-bindings/cli/app.py b/gpt4all-bindings/cli/app.py
index 6217046..95895e7 100644
--- a/gpt4all-bindings/cli/app.py
+++ b/gpt4all-bindings/cli/app.py
@@ -83,32 +83,22 @@ def repl(
             SPECIAL_COMMANDS[message](MESSAGES)
             continue

-        # if regular message, append to messages
-        MESSAGES.append({"role": "user", "content": message})
-
-        # execute chat completion and ignore the full response since 
+        # execute chat completion and ignore the full response since
         # we are outputting it incrementally
-        full_response = gpt4all_instance.chat_completion(
-            MESSAGES,
-            # preferential kwargs for chat ux
-            logits_size=0,
-            tokens_size=0,
-            n_past=0,
-            n_ctx=0,
-            n_predict=200,
-            top_k=40,
-            top_p=0.9,
-            temp=0.9,
-            n_batch=9,
-            repeat_penalty=1.1,
-            repeat_last_n=64,
-            context_erase=0.0,
-            # required kwargs for cli ux (incremental response)
-            verbose=False,
-            streaming=True,
-        )
-        # record assistant's response to messages
-        MESSAGES.append(full_response.get("choices")[0].get("message"))
+        with gpt4all_instance.chat_session():
+            full_response = gpt4all_instance.generate(
+                message,
+                n_batch=9,
+                n_predict=200,
+                repeat_last_n=64,
+                repeat_penalty=1.1,
+                temp=0.9,
+                top_k=40,
+                top_p=0.9,
+                streaming=False,
+            )
+            print(full_response)
+
         print() # newline before next prompt

[cosmic-snow]

@compilebunny Some significant changes were made to the Python bindings from v1.0.0 onwards. The CLI had to be updated for that, as well as some features reimplemented in the new bindings API.

The CLI was fixed with affd0af51fbe59276c94cb4bc0f86bebee480b42 and should now work with v1.0.2+ of the Python bindings (in addition to some 0.3.y versions).

So if you're still on v1.0.0 or v1.0.1, please update your gpt4all package and the CLI app.py.

Sorry for the inconvenience.

[FernandoFerrerTDK]

Hi! I'm getting the same error with gpt4all==1.0.3: "Error: 'GPT4All' object has no attribute 'chat_completion'" Could you please provide the Python syntax to perform a chat completion, given multiple messages? Thanks in advance!

[cosmic-snow]

Hi! I'm getting the same error with gpt4all==1.0.3: "Error: 'GPT4All' object has no attribute 'chat_completion'" Could you please provide the Python syntax to perform a chat completion, given multiple messages? Thanks in advance!

See my previous comment. chat_completion() has been removed from the Python bindings. If you're using the CLI, download a more up-to-date version of it.

[cosmic-snow]

@compilebunny Have you managed to make it work with the newer version?

[compilebunny]

@cosmic-snow Yes. The new version of app.py works fine with gpt4all 1.0.2

[PeterKatter]

I have the same problem, so i tried to install app.py as written here via "pip install App.py" But if i do this it shows the message "Using cached App.py-1.5.tar.gz (927 bytes) ERROR: App.py from https://files.pythonhosted.org/packages/92/54/dfb60298bcb00f09cb14c443f3612e2f5da39f37a1837447cbc1e03159c1/App.py-1.5.tar.gz does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found."

Otherwise i could download and extract the tar.gz, but in which folder should the extracted file be placed?

Thank you very much! :)

[cosmic-snow]

I have the same problem, so i tried to install app.py as written here via "pip install App.py"

Nowhere is there an instruction telling you to try installing it with pip. You're completely on the wrong track with that. You simply need to download the script if you want to run the CLI, then maybe install some dependencies for it. See the docs.

[PeterKatter]

Aaaah, thank you!