MikeBirdTech
commented
9 months ago You might need to add /v1 to the api_base.
What command are you running to launch Open Interpreter?
Open LanceLake opened 9 months ago
MikeBirdTech
commented
9 months ago You might need to add /v1 to the api_base.
What command are you running to launch Open Interpreter?
LanceLake
commented
9 months ago I tried it with the /v1 as well and same result.
I'm using "interpreter --local" to start it.
Notnaton
commented
9 months ago Dont add -l flag. it overwrites api_base
Once all the nessesary parameters in config is set, just use interpreter and it will load the flags from the config file
LanceLake
commented
9 months ago Ok.. That actually worked..
From OogaBooga
12:33:49-548302 ERROR Could not find the character "None" inside characters/. No character has been loaded.
Exception in ASGI applicationSo I understand what it wants. It needs a character in the JSON sent (among other things).
The JSON sent was this..
{'role': 'user', 'type': 'message', 'content': 'Open Chrome and go to YouTube.'}
It wants basically something like this..
'
{
"messages":
[
{
"role": "system",
"content": "You are a helpful assistant.",
"name": "Automated_Assistant"
},
{
"role": "user",
"content": "#Message#",
"name": "#Username#"
}
],
"mode": "chat",
"character": "Automated_Assistant",
}
'
Where would I go to manually force more key pairs to be added?
Notnaton
commented
9 months ago You need to start Oobabooga with openai compatible api, se the link for more info: https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API
LanceLake
commented
9 months ago I am. In order for it to accept the json input, it needs this format for characters.
'{
"messages": [
{
"role": "user",
"content": "Hello! Who are you?"
}
],
"mode": "chat",
"character": "Example"
}'But instruct is sending
{'role': 'user', 'type': 'message', 'content': 'Open Chrome and go to YouTube.'}
Which adds a "type" and does not include "mode" or "Character".
As per this error..
Could not find the character "None" inside characters/. No character has been loaded. Exception in ASGI application
I can see what is happening. Instruct isn't sending mode or character.
I can hard code those entries (or set them up as variables for the config, at least for character), but there are a lot of entries for the JSON and after doing a hunt and replace, I was unable to find where to do the replacement so I can have it send this instead..
'{ "messages": [ { "role": "user", "content": "Open Chrome and go to YouTube." } ], "mode": "chat", "character": "Jarvis" }'
Notnaton
commented
9 months ago I see, I shall work on a PR to fix this... We used to run Oobabooga with Open-interpreter using ooba before, but it was a buggy mess
Notnaton
commented
9 months ago Can you try to add --model openai/local
LanceLake
commented
9 months ago From instruct (Sending the same JSON as before)
Exception: Error occurred. peer closed connection without sending complete message body (incomplete chunked read)
From ooga
09:52:16-875375 ERROR Could not find the character "None" inside characters/. No character has been loaded.
Exception in ASGI application
Traceback (most recent call last):
File "E:\text-generation-webui-fresh-useable\installer_files\env\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 426, in run_asgiI see, I shall work on a PR to fix this... We used to run Oobabooga with Open-interpreter using ooba before, but it was a buggy mess
If you can just be able to add a custom JSON in the config and send that instead, that would work fine. Then no matter the system, it will be able to be configured to how it needs the JSON to be.
RPHllc
commented
9 months ago I managed to get open-interpreter to talk to text-generation-webui as follows:
I have text-generation-webui running on a separate machine with openai extension activated and a model loaded. I installed open-interpreter on a conda environment on my local machine (Mac).
I started by typing interpreter --config, to get to config.yaml The only active lines in my config.yaml are
llm.model: "gpt-4"
llm.temperature: 0
llm.api_key: sk-111111 # this is a dummy name, but without it interpreter does not work
llm.api_base: http://<url of the machine running text-generation-webui>:5000/v1 Then I start open-interpreter with: interpreter
Once the interpreter starts, type hi and enter. It should respond, in my case it says "Hello! How can I help you today?" This response is the response that came from the API as I can see it in the text-generation-webui log.
I guess I need to change the llm.model to be able to get a proper template for the model I am using. But the API interface is working.
Open-interpreter is suggesting commands to run on my computer, for example when asked to list the files on my default directory it presented "ls" and asked permission to run it. I tried it but it did not run. But this is another story...
LanceLake
commented
9 months ago I just get the same error. Are you running the most up to date Oogabooga? Can you send what is being sent in terms of JSON to ooga?
Mine is still sending this.. {'role': 'user', 'type': 'message', 'content': 'Hi'} Take a look up and it should tell you the JSON it is sending.
RPHllc
commented
9 months ago I just get the same error. Are you running the most up to date Oogabooga? Can you send what is being sent in terms of JSON to ooga?
Mine is still sending this.. {'role': 'user', 'type': 'message', 'content': 'Hi'} Take a look up and it should tell you the JSON it is sending.
I do not know how to see the JSON you showed above. By looking at the log on the oogabooga side I see:
15:34:57-831189 INFO PROMPT= [INST] hi [/INST] 15:34:58-143173 INFO GENERATE_PARAMS= { 'max_new_tokens': 32759, 'temperature': 1, 'temperature_last': False, 'dynamic_temperature': False, 'dynatemp_low': 1, 'dynatemp_high': 1, 'dynatemp_exponent': 1, 'top_p': 1, 'min_p': 0, 'top_k': 0, 'repetition_penalty': 1, 'presence_penalty': 0, 'frequency_penalty': 0, 'repetition_penalty_range': 1024, 'typical_p': 1, 'tfs': 1, 'top_a': 0, 'guidance_scale': 1, 'penalty_alpha': 0, 'mirostat_mode': 0, 'mirostat_tau': 5, 'mirostat_eta': 0.1, 'do_sample': True, 'encoder_repetition_penalty': 1, 'no_repeat_ngram_size': 0, 'min_length': 0, 'num_beams': 1, 'length_penalty': 1, 'early_stopping': False, 'use_cache': True, 'eos_token_id': [2], 'stopping_criteria': [ <modules.callbacks._StopEverythingStoppingCriteria object at 0xfffed9577df0>], 'logits_processor': []}
LanceLake
commented
9 months ago I just get the same error. Are you running the most up to date Oogabooga? Can you send what is being sent in terms of JSON to ooga? Mine is still sending this.. {'role': 'user', 'type': 'message', 'content': 'Hi'} Take a look up and it should tell you the JSON it is sending.
I do not know how to see the JSON you showed above. By looking at the log on the oogabooga side I see:
15:34:57-831189 INFO PROMPT= [INST] hi [/INST] 15:34:58-143173 INFO GENERATE_PARAMS= { 'max_new_tokens': 32759, 'temperature': 1, 'temperature_last': False, 'dynamic_temperature': False, 'dynatemp_low': 1, 'dynatemp_high': 1, 'dynatemp_exponent': 1, 'top_p': 1, 'min_p': 0, 'top_k': 0, 'repetition_penalty': 1, 'presence_penalty': 0, 'frequency_penalty': 0, 'repetition_penalty_range': 1024, 'typical_p': 1, 'tfs': 1, 'top_a': 0, 'guidance_scale': 1, 'penalty_alpha': 0, 'mirostat_mode': 0, 'mirostat_tau': 5, 'mirostat_eta': 0.1, 'do_sample': True, 'encoder_repetition_penalty': 1, 'no_repeat_ngram_size': 0, 'min_length': 0, 'num_beams': 1, 'length_penalty': 1, 'early_stopping': False, 'use_cache': True, 'eos_token_id': [2], 'stopping_criteria': [ <modules.callbacks._StopEverythingStoppingCriteria object at 0xfffed9577df0>], 'logits_processor': []}
That's the loading json.. Note how there is no message.
Scroll up on the interpreter side and you will see a big json along with what you sent. Mine looks like this..
Offline: False
Curl output: Not local
# Messages
System Message: You are Open Interpreter, a world-class programmer that can complete any goal by executing code.
First, write a plan. **Always recap the plan between each code block** (you have extreme short-term memory loss, so you need to recap the plan between each message block to retain it).
When you execute code, it will be executed **on the user's machine**. The user has given you **full and complete permission** to execute any code necessary to complete the task. Execute the code.
If you want to send data between programming languages, save the data to a txt or json.
You can access the internet. Run **any code** to achieve the goal, and if at first you don't succeed, try again and again.
You can install new packages.
When a user refers to a filename, they're likely referring to an existing file in the directory you're currently executing code in.
Write messages to the user in Markdown.
In general, try to **make plans** with as few steps as possible. As for actually executing code to carry out that plan, for *stateful* languages (like python, javascript, shell, but NOT for html which starts from 0 every time) **it's critical not to try to do everything in one code block.** You should try something, print information about it, then continue from there in tiny, informed steps. You will never get it on the first try, and attempting it in one go will often lead to errors you cant see.
You are capable of **any** task.
{'role': 'user', 'type': 'message', 'content': 'Hello'}Note how there is no "Character" key. Now look at this error from Ooga.
Could not find the character "None" inside characters/. No character has been loaded. Exception in ASGI application
What this needs is something like this JSON..
{
"messages": [
{
"role": "user",
"content": "Open Chrome and go to YouTube."
}
],
"mode": "chat",
"character": "Jarvis"
}
specifying what mode to use (chat) and what character to use (In my case, Jarvis).
I am not getting this error. This is what I see in the OI side from the time I launch it (I am using the -v option for the first time as otherwise I did not get any detail)
`(oi) ➜ interpreter -v Setting attribute verbose on openinterpreter to 'True'... Setting attribute anonymous_telemetry on openinterpreter to 'True'... Setting attribute safe_mode on openinterpreter to 'off'...
▌ Model set to openai/TheBloke_Mistral-7B-Instruct-v0.2-AWQ
Open Interpreter will require approval before running code.
Use interpreter -y to bypass this.
Press CTRL-C to exit.
`
Then I decided to make a query and this is what I got.
> What time is it? {'procedures': ["# Get calendar events\n(Mac) Usebrew install ical-buddythen something likeical-buddy eventsFrom:today to:'today+7'`", "If you encounter a traceback, don't try to use an alternative method yet. Instead:\n\nWrite a message to the user explaining what happened and theorizing why. Do not try to run_code immediatly after run_code has errored.\n\nIf a solution is apparent (and is not simply changing methods / using a new package) attempt it.\nIf not, list these steps in a message to the user, then follow them one-by-one:\n\n1. Create and run a minimal reproducible example.\n2. Use dir() to verify correct imports. There may be a better object to import from the module.\n3. Print docstrings of functions/classes using print(func.doc).\n\nOnly then are you permitted to use an alternative method."]}
We were unable to determine the context window of this model. Defaulting to 3000.
If your model can handle more, run interpreter --context_window {token limit} --max_tokens {max tokens per response}.
Continuing...
Request to litellm:
litellm.completion(model='openai/TheBloke_Mistral-7B-Instruct-v0.2-AWQ', messages=[{'role': 'system', 'content': "You are Open Interpreter, a world-class programmer that can complete any goal by executing code.\nFirst, write a plan. Always recap the plan between each code block (you have extreme short-term memory loss, so you need to recap the plan between each message block to retain it).\nWhen you execute code, it will be executed on the user's machine. The user has given you full and complete permission to execute any code necessary to complete the task. Execute the code.\nIf you want to send data between programming languages, save the data to a txt or json.\nYou can access the internet. Run any code to achieve the goal, and if at first you don't succeed, try again and again.\nYou can install new packages.\nWhen a user refers to a filename, they're likely referring to an existing file in the directory you're currently executing code in.\nWrite messages to the user in Markdown.\nIn general, try to make plans with as few steps as possible. As for actually executing code to carry out that plan, for stateful languages (like python, javascript, shell, but NOT for html which starts from 0 every time) it's critical not to try to do everything in one code block. You should try something, print information about it, then continue from there in tiny, informed steps. You will never get it on the first try, and attempting it in one go will often lead to errors you cant see.\nYou are capable of any task.\n[User Info]\nName: rp\nCWD: /Users/rp/dev/interpreter\nSHELL: /bin/zsh\nOS: Darwin\n[Recommended Procedures]\n# Get calendar events\n(Mac) Use brew install ical-buddy then something like ical-buddy eventsFrom:today to:'today+7'\n---\nIf you encounter a traceback, don't try to use an alternative method yet. Instead:\n\nWrite a message to the user explaining what happened and theorizing why. Do not try to run_code immediatly after run_code has errored.\n\nIf a solution is apparent (and is not simply changing methods / using a new package) attempt it.\nIf not, list these steps in a message to the user, then follow them one-by-one:\n\n1. Create and run a minimal reproducible example.\n2. Use dir() to verify correct imports. There may be a better object to import from the module.\n3. Print docstrings of functions/classes using print(func.doc).\n\nOnly then are you permitted to use an alternative method.\nIn your plan, include steps and, for relevant deprecation notices, EXACT CODE SNIPPETS -- these notices will VANISH once you execute your first line of code, so WRITE THEM DOWN NOW if you need them.\n\nWhen you execute code with react, your react code will be run in a script tag after being inserted into the HTML template, following the installation of React, ReactDOM, and Babel for JSX parsing. We will handle this! Don't make an HTML file to run React, just execute react.\nTo execute code on the user's machine, write a markdown code block. Specify the language after the ``. You will receive the output. Use any programming language."}, {'role': 'user', 'content': 'What time is it?'}], stream=True, api_base='http://orinnx.local:5000/v1', api_key='sk-111111')
There is much more after this, I stopped here to minimize cluttering.
LanceLake
commented
9 months ago I am not getting this error. This is what I see in the OI side from the time I launch it (I am using the -v option for the first time as otherwise I did not get any detail)
`(oi) ➜ interpreter -v Setting attribute verbose on openinterpreter to 'True'... Setting attribute anonymous_telemetry on openinterpreter to 'True'... Setting attribute safe_mode on openinterpreter to 'off'...
▌ Model set to openai/TheBloke_Mistral-7B-Instruct-v0.2-AWQ
Open Interpreter will require approval before running code.
Use interpreter -y to bypass this.
Press CTRL-C to exit.
`
Then I decided to make a query and this is what I got.
> What time is it? {'procedures': ["# Get calendar events\n(Mac) Usebrew install ical-buddythen something likeical-buddy eventsFrom:today to:'today+7'`", "If you encounter a traceback, don't try to use an alternative method yet. Instead:\n\nWrite a message to the user explaining what happened and theorizing why. Do not try to run_code immediatly after run_code has errored.\n\nIf a solution is apparent (and is not simply changing methods / using a new package) attempt it.\nIf not, list these steps in a message to the user, then follow them one-by-one:\n\n1. Create and run a minimal reproducible example.\n2. Use dir() to verify correct imports. There may be a better object to import from the module.\n3. Print docstrings of functions/classes using print(func.doc).\n\nOnly then are you permitted to use an alternative method."]}We were unable to determine the context window of this model. Defaulting to 3000.
If your model can handle more, run interpreter --context_window {token limit} --max_tokens {max tokens per response}.
Continuing...
Request to litellm: litellm.completion(model='openai/TheBloke_Mistral-7B-Instruct-v0.2-AWQ', messages=[{'role': 'system', 'content': "You are Open Interpreter, a world-class programmer that can complete any goal by executing code.\nFirst, write a plan. Always recap the plan between each code block (you have extreme short-term memory loss, so you need to recap the plan between each message block to retain it).\nWhen you execute code, it will be executed on the user's machine. The user has given you full and complete permission to execute any code necessary to complete the task. Execute the code.\nIf you want to send data between programming languages, save the data to a txt or json.\nYou can access the internet. Run any code to achieve the goal, and if at first you don't succeed, try again and again.\nYou can install new packages.\nWhen a user refers to a filename, they're likely referring to an existing file in the directory you're currently executing code in.\nWrite messages to the user in Markdown.\nIn general, try to make plans with as few steps as possible. As for actually executing code to carry out that plan, for stateful languages (like python, javascript, shell, but NOT for html which starts from 0 every time) it's critical not to try to do everything in one code block. You should try something, print information about it, then continue from there in tiny, informed steps. You will never get it on the first try, and attempting it in one go will often lead to errors you cant see.\nYou are capable of any task.\n[User Info]\nName: rp\nCWD: /Users/rp/dev/interpreter\nSHELL: /bin/zsh\nOS: Darwin\n[Recommended Procedures]\n# Get calendar events\n(Mac) Use
brew install ical-buddythen something likeical-buddy eventsFrom:today to:'today+7'\n---\nIf you encounter a traceback, don't try to use an alternative method yet. Instead:\n\nWrite a message to the user explaining what happened and theorizing why. Do not try to run_code immediatly after run_code has errored.\n\nIf a solution is apparent (and is not simply changing methods / using a new package) attempt it.\nIf not, list these steps in a message to the user, then follow them one-by-one:\n\n1. Create and run a minimal reproducible example.\n2. Use dir() to verify correct imports. There may be a better object to import from the module.\n3. Print docstrings of functions/classes using print(func.doc).\n\nOnly then are you permitted to use an alternative method.\nIn your plan, include steps and, for relevant deprecation notices, EXACT CODE SNIPPETS -- these notices will VANISH once you execute your first line of code, so WRITE THEM DOWN NOW if you need them.\n\nWhen you execute code withreact, your react code will be run in a script tag after being inserted into the HTML template, following the installation of React, ReactDOM, and Babel for JSX parsing. We will handle this! Don't make an HTML file to run React, just executereact.\nTo execute code on the user's machine, write a markdown code block. Specify the language after the``. You will receive the output. Use any programming language."}, {'role': 'user', 'content': 'What time is it?'}], stream=True, api_base='http://orinnx.local:5000/v1', api_key='sk-111111')There is much more after this, I stopped here to minimize cluttering.
RIGHT AFTER this in the JSON you are sending. :)
RPHllc
commented
9 months ago Here is the continuation:
`self.optional_params: {} kwargs[caching]: False; litellm.cache: None self.optional_params: {'stream': True, 'extra_body': {}} PROCESSED CHUNK PRE CHUNK CREATOR: ChatCompletionChunk(id='chatcmpl-1707148482206359808', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None, message={'role': 'assistant', 'content': ''})], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completions.chunk', system_fingerprint=None)
Raw OpenAI Chunk ChatCompletionChunk(id='chatcmpl-1707148482206359808', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None, message={'role': 'assistant', 'content': ''})], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completions.chunk', system_fingerprint=None)
completion obj content: model_response: ModelResponse(id='chatcmpl-7a001500-bd4a-4ec3-8db1-3bf62ae4cc4b', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content=None, role=None))], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage()); completion_obj: {'content': ''} model_response finish reason 3: None PROCESSED CHUNK POST CHUNK CREATOR: None PROCESSED CHUNK PRE CHUNK CREATOR: ChatCompletionChunk(id='chatcmpl-1707148482206359808', choices=[Choice(delta=ChoiceDelta(content='I', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None, message={'role': 'assistant', 'content': 'I'})], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completions.chunk', system_fingerprint=None)
Raw OpenAI Chunk ChatCompletionChunk(id='chatcmpl-1707148482206359808', choices=[Choice(delta=ChoiceDelta(content='I', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None, message={'role': 'assistant', 'content': 'I'})], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completions.chunk', system_fingerprint=None)
completion obj content: I
model_response: ModelResponse(id='chatcmpl-7a001500-bd4a-4ec3-8db1-3bf62ae4cc4b', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content=None, role=None))], created=1707148483, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage()); completion_obj: {'content': 'I'}
model_response finish reason 3: None
hold - False, model_response_str - I
returning model_response: ModelResponse(id='chatcmpl-1707148482206359808', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(function_call=None, tool_calls=None, content='I', role='assistant'))], created=1707148483, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage())
PROCESSED CHUNK POST CHUNK CREATOR: ModelResponse(id='chatcmpl-1707148482206359808', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(function_call=None, tool_calls=None, content='I', role='assistant'))], created=1707148483, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage())
Chunk in coding_llm ModelResponse(id='chatcmpl-1707148482206359808', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(function_call=None, tool_calls=None, content='I', role='assistant'))], created=1707148483, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage())
Chunk in terminal_interface: {'role': 'assistant', 'type': 'message', 'start': True}
Chunk in terminal_interface: {'role': 'assistant', 'type': 'message', 'content': 'I'}
PROCESSED CHUNK PRE CHUNK CREATOR: ChatCompletionChunk(id='chatcmpl-1707148482206359808',
choices=[Choice(delta=ChoiceDelta(content="'", function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0,
logprobs=None, message={'role': 'assistant', 'content': "'"})], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ',
object='chat.completions.chunk', system_fingerprint=None)
Raw OpenAI Chunk ChatCompletionChunk(id='chatcmpl-1707148482206359808', choices=[Choice(delta=ChoiceDelta(content="'", function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None, message={'role': 'assistant', 'content': "'"})], created=1707148482, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completions.chunk', system_fingerprint=None)
completion obj content: '
model_response: ModelResponse(id='chatcmpl-7a001500-bd4a-4ec3-8db1-3bf62ae4cc4b', choices=[StreamingChoices(finish_reason=None,
index=0, delta=Delta(content=None, role=None))], created=1707148483, model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ',
object='chat.completion.chunk', system_fingerprint=None, usage=Usage()); completion_obj: {'content': "'"}
model_response finish reason 3: None
hold - False, model_response_str - '
returning model_response: ModelResponse(id='chatcmpl-1707148482206359808', choices=[StreamingChoices(finish_reason=None, index=0,
delta=Delta(function_call=None, tool_calls=None, content="'", role='assistant'))], created=1707148483,
model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage())
PROCESSED CHUNK POST CHUNK CREATOR: ModelResponse(id='chatcmpl-1707148482206359808', choices=[StreamingChoices(finish_reason=None,
index=0, delta=Delta(function_call=None, tool_calls=None, content="'", role='assistant'))], created=1707148483,
model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage())
Chunk in coding_llm ModelResponse(id='chatcmpl-1707148482206359808', choices=[StreamingChoices(finish_reason=None, index=0,
delta=Delta(function_call=None, tool_calls=None, content="'", role='assistant'))], created=1707148483,
model='TheBloke_Mistral-7B-Instruct-v0.2-AWQ', object='chat.completion.chunk', system_fingerprint=None, usage=Usage())
Chunk in terminal_interface: {'role': 'assistant', 'type': 'message', 'content': "'"}
`
RPHllc
commented
9 months ago To be clear: (1) I am not using LiteLLM, (2) I realize that the model I am using may require different tuning syntax or different parameters.
LanceLake
commented
9 months ago My bad. The JSON sent was this.
{'role': 'user', 'content': 'What time is it?'}
According to the OogaBooga OpenAI API docs, you need a "character" pair and "mode" pair when sending an OpenAI request. Neither are sent. That's the issue. I could hard code it if I knew where that JSON was being generated from in the program.
Notnaton
commented
9 months ago Hey, could you try this PR? Assuming you know your way around git...
https://github.com/KillianLucas/open-interpreter/pull/955
After installing run interpreter with these flags:
-m Oobabooga/modelname
# Dont know if you need these:
-ak dummykey
-ab http://localhost:port/v1I don't know how to use git to do a pull request. However, after looking it over, it sounds like it basically sets a dummy key. I already do that. I have it set to "falsekey".
If you look on the API for oogabooga OpenAPI call page, you will see this.
curl http://127.0.0.1:5000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{
"role": "user",
"content": "Hello! Who are you?"
}
],
"mode": "chat",
"character": "Example"
}'Does this not mean you need a mode and character keypair?
Notnaton
commented
9 months ago Install from PR:
pip uninstall open-interpreter
pip install git
pip install git+https://github.com/Notnaton/open-interpreter.git@litellm-custom-providerI have updated it, now Oogabooga should work I hope
LanceLake
commented
9 months ago Nope. Same exact error.
17:37:33-823640 ERROR Could not find the character "None" inside characters/. No character has been loaded.
Exception in ASGI applicationTraceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "C:\Python311\Scripts\interpreter.exe\__main__.py", line 7, in <module>
File "C:\Python311\Lib\site-packages\interpreter\core\core.py", line 25, in start_terminal_interface
start_terminal_interface(self)
File "C:\Python311\Lib\site-packages\interpreter\terminal_interface\start_terminal_interface.py", line 684, in start_terminal_interface
interpreter.chat()
File "C:\Python311\Lib\site-packages\interpreter\core\core.py", line 86, in chat
for _ in self._streaming_chat(message=message, display=display):
File "C:\Python311\Lib\site-packages\interpreter\core\core.py", line 113, in _streaming_chat
yield from terminal_interface(self, message)
File "C:\Python311\Lib\site-packages\interpreter\terminal_interface\terminal_interface.py", line 135, in terminal_interface
for chunk in interpreter.chat(message, display=False, stream=True):
File "C:\Python311\Lib\site-packages\interpreter\core\core.py", line 148, in _streaming_chat
yield from self._respond_and_store()
File "C:\Python311\Lib\site-packages\interpreter\core\core.py", line 194, in _respond_and_store
for chunk in respond(self):
File "C:\Python311\Lib\site-packages\interpreter\core\respond.py", line 49, in respond
for chunk in interpreter.llm.run(messages_for_llm):
File "C:\Python311\Lib\site-packages\interpreter\core\llm\llm.py", line 193, in run
yield from run_text_llm(self, params)
File "C:\Python311\Lib\site-packages\interpreter\core\llm\run_text_llm.py", line 19, in run_text_llm
for chunk in llm.completions(**params):
File "C:\Python311\Lib\site-packages\interpreter\core\llm\llm.py", line 224, in fixed_litellm_completions
raise first_error
File "C:\Python311\Lib\site-packages\interpreter\core\llm\llm.py", line 205, in fixed_litellm_completions
yield from litellm.completion(**params)
File "C:\Python311\Lib\site-packages\litellm\utils.py", line 8007, in __next__
raise e
File "C:\Python311\Lib\site-packages\litellm\utils.py", line 7986, in __next__
chunk = next(self.completion_stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\openai\_streaming.py", line 40, in __next__
return self._iterator.__next__()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\openai\_streaming.py", line 55, in __stream__
for sse in iterator:
File "C:\Python311\Lib\site-packages\openai\_streaming.py", line 47, in _iter_events
yield from self._decoder.iter(self.response.iter_lines())
File "C:\Python311\Lib\site-packages\openai\_streaming.py", line 223, in iter
for line in iterator:
File "C:\Python311\Lib\site-packages\httpx\_models.py", line 857, in iter_lines
for text in self.iter_text():
File "C:\Python311\Lib\site-packages\httpx\_models.py", line 844, in iter_text
for byte_content in self.iter_bytes():
File "C:\Python311\Lib\site-packages\httpx\_models.py", line 823, in iter_bytes
for raw_bytes in self.iter_raw():
File "C:\Python311\Lib\site-packages\httpx\_models.py", line 881, in iter_raw
for raw_stream_bytes in self.stream:
File "C:\Python311\Lib\site-packages\httpx\_client.py", line 123, in __iter__
for chunk in self._stream:
File "C:\Python311\Lib\site-packages\httpx\_transports\default.py", line 103, in __iter__
with map_httpcore_exceptions():
File "C:\Python311\Lib\contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File "C:\Python311\Lib\site-packages\httpx\_transports\default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
As I tried to explain, you need a "Character" keypair in the JSON.
MikeBirdTech
commented
9 months ago OI has stated ...opinionated on the fact that if an API base is set, it needs to be OpenAI compatible
Is having a Character keypair OpenAI compatible?
LanceLake
commented
9 months ago OI has stated
...opinionated on the fact that if an API base is set, it needs to be OpenAI compatibleIs having a
Characterkeypair OpenAI compatible?
No. But how do you explain the error I get then over and over saying it can't load a character?
Let me try setting a character called "None" and see if that works.
LanceLake
commented
9 months ago That works. Not the best solution, but one that I can work with. Thank you. :)
userbox020
commented
9 months ago sup bro, im also using ooba openai api as backend for my local llms
the follow works for me:
1. start ooba with the flags --api --listen
2. load the model
3. start code interpreterinterpreter_chat.py
from interpreter import interpreter
#interpreter.offline = True # Disables online features like Open Procedures
interpreter.llm.model = "openai/x" # Tells OI to send messages in OpenAI's format
interpreter.llm.api_key = "fake_key" # LiteLLM, which we use to talk to LM Studio, requires this
interpreter.llm.api_base = "http://192.168.0.169:5000/v1" # Point this at any OpenAI compatible server
interpreter.chat()You just change your ooba server ip, in my case is 192.168.0.169:5000
By the way, what llm are you using for code interpreter?
rosx27
commented
7 months ago i'm having the same issue when using oobabooga. i have used --api and --listen arguments but still getting an error, specifically AttributeError: 'NoneType' object has no attribute 'replace' from this logs:
Exception in ASGI application
Traceback (most recent call last):
File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\sse_starlette\sse.py", line 247, in __call__
await wrap(partial(self.listen_for_disconnect, receive))
File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\sse_starlette\sse.py", line 236, in wrap
await func()
File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\sse_starlette\sse.py", line 191, in listen_for_disconnect
message = await receive()
^^^^^^^^^^^^^^^
File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 571, in receive
await self.message_event.wait()
File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\asyncio\locks.py", line 213, in wait
await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 26baeb55a90
During handling of the above exception, another exception occurred:
+ Exception Group Traceback (most recent call last):
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 412, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\fastapi\applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\applications.py", line 123, in __call__
| await self.middleware_stack(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\middleware\errors.py", line 186, in __call__
| raise exc
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\middleware\errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\middleware\cors.py", line 83, in __call__
| await self.app(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\middleware\exceptions.py", line 62, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\routing.py", line 758, in __call__
| await self.middleware_stack(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\routing.py", line 778, in app
| await route.handle(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\routing.py", line 299, in handle
| await self.app(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\routing.py", line 79, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
| raise exc
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
| await app(scope, receive, sender)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\starlette\routing.py", line 77, in app
| await response(scope, receive, send)
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\sse_starlette\sse.py", line 233, in __call__
| async with anyio.create_task_group() as task_group:
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\anyio\_backends\_asyncio.py", line 678, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\sse_starlette\sse.py", line 236, in wrap
| await func()
| File "D:\AI\Language\text-generation-webui\installer_files\env\Lib\site-packages\sse_starlette\sse.py", line 221, in stream_response
| async for data in self.body_iterator:
| File "D:\AI\Language\text-generation-webui\extensions\openai\script.py", line 127, in generator
| for resp in response:
| File "D:\AI\Language\text-generation-webui\extensions\openai\completions.py", line 540, in stream_chat_completions
| for resp in chat_completions_common(body, is_legacy, stream=True):
| File "D:\AI\Language\text-generation-webui\extensions\openai\completions.py", line 315, in chat_completions_common
| prompt = generate_chat_prompt(user_input, generate_params)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "D:\AI\Language\text-generation-webui\modules\chat.py", line 97, in generate_chat_prompt
| user_bio=replace_character_names(state['user_bio'], state['name1'], state['name2']),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "D:\AI\Language\text-generation-webui\modules\chat.py", line 636, in replace_character_names
| text = text.replace('{{user}}', name1).replace('{{char}}', name2)
| ^^^^^^^^^^^^
| AttributeError: 'NoneType' object has no attribute 'replace'i tried using koboldcpp for sanity check and its works just fine. however, i prefer using text-generation-webui so i can fully utilize my GPU.
EDIT - 28 Mar: i found similar issue and potential solution on oobabooga, i just dont know how can we modify the http request.
Describe the bug
This is my configuration file.
The system for OogaBooga uses OpenAPI at 0.0.0.0:5001. I tried this one and got the same error.
I presume that perhaps OogaBooga doesn't accept curl? How would I set things up to use OogaBooga?
Reproduce
Expected behavior
Should work.
Screenshots
No response
Open Interpreter version
0.2.0
Python version
3.11.5
Operating System name and version
Windows 10
Additional context
No response