search

szczyglis-dev / py-gpt

Desktop AI Assistant powered by GPT-4, GPT-4 Vision, GPT-3.5, DALL-E 3, Langchain, Llama-index, chat, vision, voice control, image generation and analysis, autonomous agents, code and command execution, file upload and download, speech synthesis and recognition, access to Web, memory, prompt presets, plugins, assistants & more. Linux, Windows, Mac.
https://pygpt.net
MIT License
449 stars 92 forks source link

web search does not work #20

Closed bradjohnl closed 4 months ago

bradjohnl commented 4 months ago

When I try to use the web search, it writes the command but it doesn't actually perform the search:

15:08:29: > Search the web for the next SpaceX events
15:08:32: 
{"cmd": "web", "params": {"query": "next SpaceX events"}}
# Nothing here...

Logs with debug enabled:

....
....
EVENT AFTER: {"name": "enable", "data": {"value": "openai_dalle"}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: plugin.option.get
EVENT BEFORE: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Apply [plugin.option.get] to plugin: audio_openai_whisper
Apply [plugin.option.get] to plugin: cmd_web_google
Apply [plugin.option.get] to plugin: cmd_files
Apply [plugin.option.get] to plugin: cmd_code_interpreter
Apply [plugin.option.get] to plugin: cmd_custom
Apply [plugin.option.get] to plugin: cmd_history
Apply [plugin.option.get] to plugin: openai_dalle
Apply [plugin.option.get] to plugin: idx_llama_index
Apply [plugin.option.get] to plugin: crontab
Dispatch event end: plugin.option.get
EVENT AFTER: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.vision
EVENT BEFORE: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.vision] to plugin: audio_openai_whisper
Apply [ui.vision] to plugin: cmd_web_google
Apply [ui.vision] to plugin: cmd_files
Apply [ui.vision] to plugin: cmd_code_interpreter
Apply [ui.vision] to plugin: cmd_custom
Apply [ui.vision] to plugin: cmd_history
Apply [ui.vision] to plugin: openai_dalle
Apply [ui.vision] to plugin: idx_llama_index
Apply [ui.vision] to plugin: crontab
Dispatch event end: ui.vision
EVENT AFTER: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.attachments
EVENT BEFORE: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.attachments] to plugin: audio_openai_whisper
Apply [ui.attachments] to plugin: cmd_web_google
Apply [ui.attachments] to plugin: cmd_files
Apply [ui.attachments] to plugin: cmd_code_interpreter
Apply [ui.attachments] to plugin: cmd_custom
Apply [ui.attachments] to plugin: cmd_history
Apply [ui.attachments] to plugin: openai_dalle
Apply [ui.attachments] to plugin: idx_llama_index
Apply [ui.attachments] to plugin: crontab
Dispatch event end: ui.attachments
EVENT AFTER: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: enable
EVENT BEFORE: {"name": "enable", "data": {"value": "openai_vision"}, "ctx": null, "stop": false, "internal": false}
Apply [enable] to plugin: audio_openai_whisper
Apply [enable] to plugin: cmd_web_google
Apply [enable] to plugin: cmd_files
Apply [enable] to plugin: cmd_code_interpreter
Apply [enable] to plugin: cmd_custom
Apply [enable] to plugin: cmd_history
Apply [enable] to plugin: openai_dalle
Apply [enable] to plugin: openai_vision
Apply [enable] to plugin: idx_llama_index
Apply [enable] to plugin: crontab
Dispatch event end: enable
EVENT AFTER: {"name": "enable", "data": {"value": "openai_vision"}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: plugin.option.get
EVENT BEFORE: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Apply [plugin.option.get] to plugin: audio_openai_whisper
Apply [plugin.option.get] to plugin: cmd_web_google
Apply [plugin.option.get] to plugin: cmd_files
Apply [plugin.option.get] to plugin: cmd_code_interpreter
Apply [plugin.option.get] to plugin: cmd_custom
Apply [plugin.option.get] to plugin: cmd_history
Apply [plugin.option.get] to plugin: openai_dalle
Apply [plugin.option.get] to plugin: openai_vision
Apply [plugin.option.get] to plugin: idx_llama_index
Apply [plugin.option.get] to plugin: crontab
Dispatch event end: plugin.option.get
EVENT AFTER: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.vision
EVENT BEFORE: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.vision] to plugin: audio_openai_whisper
Apply [ui.vision] to plugin: cmd_web_google
Apply [ui.vision] to plugin: cmd_files
Apply [ui.vision] to plugin: cmd_code_interpreter
Apply [ui.vision] to plugin: cmd_custom
Apply [ui.vision] to plugin: cmd_history
Apply [ui.vision] to plugin: openai_dalle
Apply [ui.vision] to plugin: openai_vision
Apply [ui.vision] to plugin: idx_llama_index
Apply [ui.vision] to plugin: crontab
Dispatch event end: ui.vision
EVENT AFTER: {"name": "ui.vision", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.attachments
EVENT BEFORE: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.attachments] to plugin: audio_openai_whisper
Apply [ui.attachments] to plugin: cmd_web_google
Apply [ui.attachments] to plugin: cmd_files
Apply [ui.attachments] to plugin: cmd_code_interpreter
Apply [ui.attachments] to plugin: cmd_custom
Apply [ui.attachments] to plugin: cmd_history
Apply [ui.attachments] to plugin: openai_dalle
Apply [ui.attachments] to plugin: openai_vision
Apply [ui.attachments] to plugin: idx_llama_index
Apply [ui.attachments] to plugin: crontab
Dispatch event end: ui.attachments
EVENT AFTER: {"name": "ui.attachments", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: enable
EVENT BEFORE: {"name": "enable", "data": {"value": "real_time"}, "ctx": null, "stop": false, "internal": false}
Apply [enable] to plugin: real_time
Apply [enable] to plugin: audio_openai_whisper
Apply [enable] to plugin: cmd_web_google
Apply [enable] to plugin: cmd_files
Apply [enable] to plugin: cmd_code_interpreter
Apply [enable] to plugin: cmd_custom
Apply [enable] to plugin: cmd_history
Apply [enable] to plugin: openai_dalle
Apply [enable] to plugin: openai_vision
Apply [enable] to plugin: idx_llama_index
Apply [enable] to plugin: crontab
Dispatch event end: enable
EVENT AFTER: {"name": "enable", "data": {"value": "real_time"}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: plugin.option.get
EVENT BEFORE: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Apply [plugin.option.get] to plugin: real_time
Apply [plugin.option.get] to plugin: audio_openai_whisper
Apply [plugin.option.get] to plugin: cmd_web_google
Apply [plugin.option.get] to plugin: cmd_files
Apply [plugin.option.get] to plugin: cmd_code_interpreter
Apply [plugin.option.get] to plugin: cmd_custom
Apply [plugin.option.get] to plugin: cmd_history
Apply [plugin.option.get] to plugin: openai_dalle
Apply [plugin.option.get] to plugin: openai_vision
Apply [plugin.option.get] to plugin: idx_llama_index
Apply [plugin.option.get] to plugin: crontab
Dispatch event end: plugin.option.get
EVENT AFTER: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.vision
EVENT BEFORE: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.vision] to plugin: real_time
Apply [ui.vision] to plugin: audio_openai_whisper
Apply [ui.vision] to plugin: cmd_web_google
Apply [ui.vision] to plugin: cmd_files
Apply [ui.vision] to plugin: cmd_code_interpreter
Apply [ui.vision] to plugin: cmd_custom
Apply [ui.vision] to plugin: cmd_history
Apply [ui.vision] to plugin: openai_dalle
Apply [ui.vision] to plugin: openai_vision
Apply [ui.vision] to plugin: idx_llama_index
Apply [ui.vision] to plugin: crontab
Dispatch event end: ui.vision
EVENT AFTER: {"name": "ui.vision", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.attachments
EVENT BEFORE: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.attachments] to plugin: real_time
Apply [ui.attachments] to plugin: audio_openai_whisper
Apply [ui.attachments] to plugin: cmd_web_google
Apply [ui.attachments] to plugin: cmd_files
Apply [ui.attachments] to plugin: cmd_code_interpreter
Apply [ui.attachments] to plugin: cmd_custom
Apply [ui.attachments] to plugin: cmd_history
Apply [ui.attachments] to plugin: openai_dalle
Apply [ui.attachments] to plugin: openai_vision
Apply [ui.attachments] to plugin: idx_llama_index
Apply [ui.attachments] to plugin: crontab
Dispatch event end: ui.attachments
EVENT AFTER: {"name": "ui.attachments", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: plugin.option.get
EVENT BEFORE: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Apply [plugin.option.get] to plugin: real_time
Apply [plugin.option.get] to plugin: audio_openai_whisper
Apply [plugin.option.get] to plugin: cmd_web_google
Apply [plugin.option.get] to plugin: cmd_files
Apply [plugin.option.get] to plugin: cmd_code_interpreter
Apply [plugin.option.get] to plugin: cmd_custom
Apply [plugin.option.get] to plugin: cmd_history
Apply [plugin.option.get] to plugin: openai_dalle
Apply [plugin.option.get] to plugin: openai_vision
Apply [plugin.option.get] to plugin: idx_llama_index
Apply [plugin.option.get] to plugin: crontab
Dispatch event end: plugin.option.get
EVENT AFTER: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.vision
EVENT BEFORE: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.vision] to plugin: real_time
Apply [ui.vision] to plugin: audio_openai_whisper
Apply [ui.vision] to plugin: cmd_web_google
Apply [ui.vision] to plugin: cmd_files
Apply [ui.vision] to plugin: cmd_code_interpreter
Apply [ui.vision] to plugin: cmd_custom
Apply [ui.vision] to plugin: cmd_history
Apply [ui.vision] to plugin: openai_dalle
Apply [ui.vision] to plugin: openai_vision
Apply [ui.vision] to plugin: idx_llama_index
Apply [ui.vision] to plugin: crontab
Dispatch event end: ui.vision
EVENT AFTER: {"name": "ui.vision", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.attachments
EVENT BEFORE: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.attachments] to plugin: real_time
Apply [ui.attachments] to plugin: audio_openai_whisper
Apply [ui.attachments] to plugin: cmd_web_google
Apply [ui.attachments] to plugin: cmd_files
Apply [ui.attachments] to plugin: cmd_code_interpreter
Apply [ui.attachments] to plugin: cmd_custom
Apply [ui.attachments] to plugin: cmd_history
Apply [ui.attachments] to plugin: openai_dalle
Apply [ui.attachments] to plugin: openai_vision
Apply [ui.attachments] to plugin: idx_llama_index
Apply [ui.attachments] to plugin: crontab
Dispatch event end: ui.attachments
EVENT AFTER: {"name": "ui.attachments", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Checking for updates...
Type: URLError, Message: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)>
Traceback:
  File "urllib/request.py", line 536, in _open
  File "urllib/request.py", line 496, in _call_chain
  File "urllib/request.py", line 1391, in https_open
  File "urllib/request.py", line 1351, in do_open

Failed to check for updates
No updates available.
Dispatch event begin: plugin.option.get
EVENT BEFORE: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Apply [plugin.option.get] to plugin: real_time
Apply [plugin.option.get] to plugin: audio_openai_whisper
Apply [plugin.option.get] to plugin: cmd_web_google
Apply [plugin.option.get] to plugin: cmd_files
Apply [plugin.option.get] to plugin: cmd_code_interpreter
Apply [plugin.option.get] to plugin: cmd_custom
Apply [plugin.option.get] to plugin: cmd_history
Apply [plugin.option.get] to plugin: openai_dalle
Apply [plugin.option.get] to plugin: openai_vision
Apply [plugin.option.get] to plugin: idx_llama_index
Apply [plugin.option.get] to plugin: crontab
Dispatch event end: plugin.option.get
EVENT AFTER: {"name": "plugin.option.get", "data": {"name": "scheduled_tasks_count", "value": 0}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.vision
EVENT BEFORE: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.vision] to plugin: real_time
Apply [ui.vision] to plugin: audio_openai_whisper
Apply [ui.vision] to plugin: cmd_web_google
Apply [ui.vision] to plugin: cmd_files
Apply [ui.vision] to plugin: cmd_code_interpreter
Apply [ui.vision] to plugin: cmd_custom
Apply [ui.vision] to plugin: cmd_history
Apply [ui.vision] to plugin: openai_dalle
Apply [ui.vision] to plugin: openai_vision
Apply [ui.vision] to plugin: idx_llama_index
Apply [ui.vision] to plugin: crontab
Dispatch event end: ui.vision
EVENT AFTER: {"name": "ui.vision", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.attachments
EVENT BEFORE: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.attachments] to plugin: real_time
Apply [ui.attachments] to plugin: audio_openai_whisper
Apply [ui.attachments] to plugin: cmd_web_google
Apply [ui.attachments] to plugin: cmd_files
Apply [ui.attachments] to plugin: cmd_code_interpreter
Apply [ui.attachments] to plugin: cmd_custom
Apply [ui.attachments] to plugin: cmd_history
Apply [ui.attachments] to plugin: openai_dalle
Apply [ui.attachments] to plugin: openai_vision
Apply [ui.attachments] to plugin: idx_llama_index
Apply [ui.attachments] to plugin: crontab
Dispatch event end: ui.attachments
EVENT AFTER: {"name": "ui.attachments", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.vision
EVENT BEFORE: {"name": "ui.vision", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.vision] to plugin: real_time
Apply [ui.vision] to plugin: audio_openai_whisper
Apply [ui.vision] to plugin: cmd_web_google
Apply [ui.vision] to plugin: cmd_files
Apply [ui.vision] to plugin: cmd_code_interpreter
Apply [ui.vision] to plugin: cmd_custom
Apply [ui.vision] to plugin: cmd_history
Apply [ui.vision] to plugin: openai_dalle
Apply [ui.vision] to plugin: openai_vision
Apply [ui.vision] to plugin: idx_llama_index
Apply [ui.vision] to plugin: crontab
Dispatch event end: ui.vision
EVENT AFTER: {"name": "ui.vision", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ui.attachments
EVENT BEFORE: {"name": "ui.attachments", "data": {"mode": "chat", "value": false}, "ctx": null, "stop": false, "internal": false}
Apply [ui.attachments] to plugin: real_time
Apply [ui.attachments] to plugin: audio_openai_whisper
Apply [ui.attachments] to plugin: cmd_web_google
Apply [ui.attachments] to plugin: cmd_files
Apply [ui.attachments] to plugin: cmd_code_interpreter
Apply [ui.attachments] to plugin: cmd_custom
Apply [ui.attachments] to plugin: cmd_history
Apply [ui.attachments] to plugin: openai_dalle
Apply [ui.attachments] to plugin: openai_vision
Apply [ui.attachments] to plugin: idx_llama_index
Apply [ui.attachments] to plugin: crontab
Dispatch event end: ui.attachments
EVENT AFTER: {"name": "ui.attachments", "data": {"mode": "chat", "value": true}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: user.send
EVENT BEFORE: {"name": "user.send", "data": {"value": "Search the web for the next SpaceX events"}, "ctx": null, "stop": false, "internal": false}
Apply [user.send] to plugin: real_time
Apply [user.send] to plugin: audio_openai_whisper
Apply [user.send] to plugin: cmd_web_google
Apply [user.send] to plugin: cmd_files
Apply [user.send] to plugin: cmd_code_interpreter
Apply [user.send] to plugin: cmd_custom
Apply [user.send] to plugin: cmd_history
Apply [user.send] to plugin: openai_dalle
Apply [user.send] to plugin: openai_vision
Apply [user.send] to plugin: idx_llama_index
Apply [user.send] to plugin: crontab
Dispatch event end: user.send
EVENT AFTER: {"name": "user.send", "data": {"value": "Search the web for the next SpaceX events"}, "ctx": null, "stop": false, "internal": false}
Begin.
Input prompt: Search the web for the next SpaceX events
Dispatch event begin: input.before
EVENT BEFORE: {"name": "input.before", "data": {"value": "Search the web for the next SpaceX events", "mode": "chat"}, "ctx": null, "stop": false, "internal": false}
Apply [input.before] to plugin: real_time
Apply [input.before] to plugin: audio_openai_whisper
Apply [input.before] to plugin: cmd_web_google
Apply [input.before] to plugin: cmd_files
Apply [input.before] to plugin: cmd_code_interpreter
Apply [input.before] to plugin: cmd_custom
Apply [input.before] to plugin: cmd_history
Apply [input.before] to plugin: openai_dalle
Apply [input.before] to plugin: openai_vision
Apply [input.before] to plugin: idx_llama_index
Apply [input.before] to plugin: crontab
Dispatch event end: input.before
EVENT AFTER: {"name": "input.before", "data": {"value": "Search the web for the next SpaceX events", "mode": "chat"}, "ctx": null, "stop": false, "internal": false}
User name: 
Dispatch event begin: user.name
EVENT BEFORE: {"name": "user.name", "data": {"value": ""}, "ctx": null, "stop": false, "internal": false}
Apply [user.name] to plugin: real_time
Apply [user.name] to plugin: audio_openai_whisper
Apply [user.name] to plugin: cmd_web_google
Apply [user.name] to plugin: cmd_files
Apply [user.name] to plugin: cmd_code_interpreter
Apply [user.name] to plugin: cmd_custom
Apply [user.name] to plugin: cmd_history
Apply [user.name] to plugin: openai_dalle
Apply [user.name] to plugin: openai_vision
Apply [user.name] to plugin: idx_llama_index
Apply [user.name] to plugin: crontab
Dispatch event end: user.name
EVENT AFTER: {"name": "user.name", "data": {"value": ""}, "ctx": null, "stop": false, "internal": false}
Dispatch event begin: ai.name
EVENT BEFORE: {"name": "ai.name", "data": {"value": ""}, "ctx": null, "stop": false, "internal": false}
Apply [ai.name] to plugin: real_time
Apply [ai.name] to plugin: audio_openai_whisper
Apply [ai.name] to plugin: cmd_web_google
Apply [ai.name] to plugin: cmd_files
Apply [ai.name] to plugin: cmd_code_interpreter
Apply [ai.name] to plugin: cmd_custom
Apply [ai.name] to plugin: cmd_history
Apply [ai.name] to plugin: openai_dalle
Apply [ai.name] to plugin: openai_vision
Apply [ai.name] to plugin: idx_llama_index
Apply [ai.name] to plugin: crontab
Dispatch event end: ai.name
EVENT AFTER: {"name": "ai.name", "data": {"value": ""}, "ctx": null, "stop": false, "internal": false}
Context: INPUT: {"id": null, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}
Dispatch event begin: ctx.before
EVENT BEFORE: {"name": "ctx.before", "data": null, "ctx": {"id": null, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}, "stop": false, "internal": false}
Apply [ctx.before] to plugin: real_time
Apply [ctx.before] to plugin: audio_openai_whisper
Apply [ctx.before] to plugin: cmd_web_google
Apply [ctx.before] to plugin: cmd_files
Apply [ctx.before] to plugin: cmd_code_interpreter
Apply [ctx.before] to plugin: cmd_custom
Apply [ctx.before] to plugin: cmd_history
Apply [ctx.before] to plugin: openai_dalle
Apply [ctx.before] to plugin: openai_vision
Apply [ctx.before] to plugin: idx_llama_index
Apply [ctx.before] to plugin: crontab
Dispatch event end: ctx.before
EVENT AFTER: {"name": "ctx.before", "data": null, "ctx": {"id": null, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}, "stop": false, "internal": false}
Dispatch event begin: pre.prompt
EVENT BEFORE: {"name": "pre.prompt", "data": {"mode": "chat", "value": "You are a helpful assistant."}, "ctx": null, "stop": false, "internal": false}
Apply [pre.prompt] to plugin: real_time
Apply [pre.prompt] to plugin: audio_openai_whisper
Apply [pre.prompt] to plugin: cmd_web_google
Apply [pre.prompt] to plugin: cmd_files
Apply [pre.prompt] to plugin: cmd_code_interpreter
Apply [pre.prompt] to plugin: cmd_custom
Apply [pre.prompt] to plugin: cmd_history
Apply [pre.prompt] to plugin: openai_dalle
Apply [pre.prompt] to plugin: openai_vision
Apply [pre.prompt] to plugin: idx_llama_index
Apply [pre.prompt] to plugin: crontab
Dispatch event end: pre.prompt
EVENT AFTER: {"name": "pre.prompt", "data": {"mode": "chat", "value": "You are a helpful assistant."}, "ctx": null, "stop": false, "internal": false}
Plugin: real_time:on_system_prompt [before]: You are a helpful assistant.
Plugin: real_time:on_system_prompt [after]: You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.
Dispatch event begin: post.prompt
EVENT BEFORE: {"name": "post.prompt", "data": {"mode": "chat", "value": "You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.\nIMAGE GENERATION: Whenever I provide a basic idea or concept for an image, such as 'a picture of mountains', I want you to ALWAYS translate it into English and expand and elaborate on this idea. Use your  knowledge and creativity to add details that would make the image more vivid and interesting. This could include specifying the time of day, weather conditions, surrounding environment, and any additional elements that could enhance the scene. Your goal is to create a detailed and descriptive prompt that provides DALL-E  with enough information to generate a rich and visually appealing image. Remember to maintain the original  intent of my request while enriching the description with your imaginative details. HOW TO START IMAGE GENERATION: to start image generation return to me prepared prompt in JSON format, all in one line,  using following syntax: ~###~{\"cmd\": \"image\", \"params\": {\"query\": \"your query here\"}}~###~. Use ONLY this syntax and remember to surround JSON string with ~###~. DO NOT use any other syntax. Use English in the generated JSON command, but conduct all the remaining parts of the discussion with me in the language in which I am speaking to you. The image will be generated on my machine  immediately after the command is issued, allowing us to discuss the photo once it has been created.  Please engage with me about the photo itself, not only by giving the generate command. \nADDITIONAL CONTEXT: I will provide you with additional data about my question. When it is provided, then use this data as your additional knowledge and use it in your response. Additional context will be prefixed with an \"Additional data:\" prefix. You can also provide a command to query my knowledge database anytime you need any additional data - to do this, return to me the prepared prompt in JSON format, all in one line, using the following syntax: ~###~{\"cmd\": \"get_knowledge\", \"params\": {\"question\": \"simple and concrete question here\"}}~###~. Use ONLY this syntax and remember to surround the JSON string with ~###~. DO NOT use any other syntax. When making query use language that I spoke to you."}, "ctx": {"id": null, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}, "stop": false, "internal": false}
Apply [post.prompt] to plugin: real_time
Apply [post.prompt] to plugin: audio_openai_whisper
Apply [post.prompt] to plugin: cmd_web_google
Apply [post.prompt] to plugin: cmd_files
Apply [post.prompt] to plugin: cmd_code_interpreter
Apply [post.prompt] to plugin: cmd_custom
Apply [post.prompt] to plugin: cmd_history
Apply [post.prompt] to plugin: openai_dalle
Apply [post.prompt] to plugin: openai_vision
Apply [post.prompt] to plugin: idx_llama_index
Apply [post.prompt] to plugin: crontab
Dispatch event end: post.prompt
EVENT AFTER: {"name": "post.prompt", "data": {"mode": "chat", "value": "You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.\nIMAGE GENERATION: Whenever I provide a basic idea or concept for an image, such as 'a picture of mountains', I want you to ALWAYS translate it into English and expand and elaborate on this idea. Use your  knowledge and creativity to add details that would make the image more vivid and interesting. This could include specifying the time of day, weather conditions, surrounding environment, and any additional elements that could enhance the scene. Your goal is to create a detailed and descriptive prompt that provides DALL-E  with enough information to generate a rich and visually appealing image. Remember to maintain the original  intent of my request while enriching the description with your imaginative details. HOW TO START IMAGE GENERATION: to start image generation return to me prepared prompt in JSON format, all in one line,  using following syntax: ~###~{\"cmd\": \"image\", \"params\": {\"query\": \"your query here\"}}~###~. Use ONLY this syntax and remember to surround JSON string with ~###~. DO NOT use any other syntax. Use English in the generated JSON command, but conduct all the remaining parts of the discussion with me in the language in which I am speaking to you. The image will be generated on my machine  immediately after the command is issued, allowing us to discuss the photo once it has been created.  Please engage with me about the photo itself, not only by giving the generate command. \nADDITIONAL CONTEXT: I will provide you with additional data about my question. When it is provided, then use this data as your additional knowledge and use it in your response. Additional context will be prefixed with an \"Additional data:\" prefix. You can also provide a command to query my knowledge database anytime you need any additional data - to do this, return to me the prepared prompt in JSON format, all in one line, using the following syntax: ~###~{\"cmd\": \"get_knowledge\", \"params\": {\"question\": \"simple and concrete question here\"}}~###~. Use ONLY this syntax and remember to surround the JSON string with ~###~. DO NOT use any other syntax. When making query use language that I spoke to you."}, "ctx": {"id": null, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}, "stop": false, "internal": false}
Dispatch event begin: cmd.syntax.inline
EVENT BEFORE: {"name": "cmd.syntax.inline", "data": {"mode": "chat", "prompt": "You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.\nIMAGE GENERATION: Whenever I provide a basic idea or concept for an image, such as 'a picture of mountains', I want you to ALWAYS translate it into English and expand and elaborate on this idea. Use your  knowledge and creativity to add details that would make the image more vivid and interesting. This could include specifying the time of day, weather conditions, surrounding environment, and any additional elements that could enhance the scene. Your goal is to create a detailed and descriptive prompt that provides DALL-E  with enough information to generate a rich and visually appealing image. Remember to maintain the original  intent of my request while enriching the description with your imaginative details. HOW TO START IMAGE GENERATION: to start image generation return to me prepared prompt in JSON format, all in one line,  using following syntax: ~###~{\"cmd\": \"image\", \"params\": {\"query\": \"your query here\"}}~###~. Use ONLY this syntax and remember to surround JSON string with ~###~. DO NOT use any other syntax. Use English in the generated JSON command, but conduct all the remaining parts of the discussion with me in the language in which I am speaking to you. The image will be generated on my machine  immediately after the command is issued, allowing us to discuss the photo once it has been created.  Please engage with me about the photo itself, not only by giving the generate command. \nADDITIONAL CONTEXT: I will provide you with additional data about my question. When it is provided, then use this data as your additional knowledge and use it in your response. Additional context will be prefixed with an \"Additional data:\" prefix. You can also provide a command to query my knowledge database anytime you need any additional data - to do this, return to me the prepared prompt in JSON format, all in one line, using the following syntax: ~###~{\"cmd\": \"get_knowledge\", \"params\": {\"question\": \"simple and concrete question here\"}}~###~. Use ONLY this syntax and remember to surround the JSON string with ~###~. DO NOT use any other syntax. When making query use language that I spoke to you. RUNNING COMMANDS:\nYou can execute commands and also use them to run commands in the user's environment.\n\nImportant rules:\n1) The list of available commands is defined below.\n2) To execute a defined command, return a JSON object with the \"cmd\" key and the command name as its value.\n3) Always use the syntax defined in the command definition and the correct command name.\n4) Put command parameters in the \"params\" key. Example: {\"cmd\": \"web\", \"params\": {\"query\": \"some query\"}}. Use ONLY this syntax. DO NOT use any other syntax.\n5) Append the JSON object to the response at the end and around it with the `~###~` characters. Example: text response ~###~ {\"cmd\": \"web\", \"params\": {\"query\": \"some query\"}} ~###~.\n6) If you want to execute a command without any response, return only the JSON object.\n7) Responses from commands will be returned in the \"result\" key.\n8) Commands are listed one command per line and each command is described with syntax: \"<name>\": <action>, params: <params>\n9) Always use the correct command name, e.g., if the command name is \"sys_exec\", then use \"sys_exec\" and don't use other names, like \"run\" or something.\n10) With these commands, you are allowed to run external commands and apps in the user's system (environment).\n11) Always use the defined syntax to prevent errors.\n12) Always choose the most appropriate command from the list to perform the task, based on the description of the action performed by a given command.\n13) Reply to the user in the language in which they started the conversation with you.\n14) Use ONLY params described in the command definition, do NOT use any additional params not described on the list.\n15) ALWAYS remember that any text content must appear at the beginning of your response and commands must only be included at the end.\n16) Every command parameter must be placed in one line, so when you generate code you must put all of the code in one line.\n17) Run the commands immediately.\n\nCommands list:", "syntax": []}, "ctx": null, "stop": false, "internal": false}
Apply [cmd.syntax.inline] to plugin: real_time
Apply [cmd.syntax.inline] to plugin: audio_openai_whisper
Apply [cmd.syntax.inline] to plugin: cmd_web_google
Apply [cmd.syntax.inline] to plugin: cmd_files
Apply [cmd.syntax.inline] to plugin: cmd_code_interpreter
Apply [cmd.syntax.inline] to plugin: cmd_custom
Apply [cmd.syntax.inline] to plugin: cmd_history
Apply [cmd.syntax.inline] to plugin: openai_dalle
Apply [cmd.syntax.inline] to plugin: openai_vision
Apply [cmd.syntax.inline] to plugin: idx_llama_index
Apply [cmd.syntax.inline] to plugin: crontab
Dispatch event end: cmd.syntax.inline
EVENT AFTER: {"name": "cmd.syntax.inline", "data": {"mode": "chat", "prompt": "You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.\nIMAGE GENERATION: Whenever I provide a basic idea or concept for an image, such as 'a picture of mountains', I want you to ALWAYS translate it into English and expand and elaborate on this idea. Use your  knowledge and creativity to add details that would make the image more vivid and interesting. This could include specifying the time of day, weather conditions, surrounding environment, and any additional elements that could enhance the scene. Your goal is to create a detailed and descriptive prompt that provides DALL-E  with enough information to generate a rich and visually appealing image. Remember to maintain the original  intent of my request while enriching the description with your imaginative details. HOW TO START IMAGE GENERATION: to start image generation return to me prepared prompt in JSON format, all in one line,  using following syntax: ~###~{\"cmd\": \"image\", \"params\": {\"query\": \"your query here\"}}~###~. Use ONLY this syntax and remember to surround JSON string with ~###~. DO NOT use any other syntax. Use English in the generated JSON command, but conduct all the remaining parts of the discussion with me in the language in which I am speaking to you. The image will be generated on my machine  immediately after the command is issued, allowing us to discuss the photo once it has been created.  Please engage with me about the photo itself, not only by giving the generate command. \nADDITIONAL CONTEXT: I will provide you with additional data about my question. When it is provided, then use this data as your additional knowledge and use it in your response. Additional context will be prefixed with an \"Additional data:\" prefix. You can also provide a command to query my knowledge database anytime you need any additional data - to do this, return to me the prepared prompt in JSON format, all in one line, using the following syntax: ~###~{\"cmd\": \"get_knowledge\", \"params\": {\"question\": \"simple and concrete question here\"}}~###~. Use ONLY this syntax and remember to surround the JSON string with ~###~. DO NOT use any other syntax. When making query use language that I spoke to you. RUNNING COMMANDS:\nYou can execute commands and also use them to run commands in the user's environment.\n\nImportant rules:\n1) The list of available commands is defined below.\n2) To execute a defined command, return a JSON object with the \"cmd\" key and the command name as its value.\n3) Always use the syntax defined in the command definition and the correct command name.\n4) Put command parameters in the \"params\" key. Example: {\"cmd\": \"web\", \"params\": {\"query\": \"some query\"}}. Use ONLY this syntax. DO NOT use any other syntax.\n5) Append the JSON object to the response at the end and around it with the `~###~` characters. Example: text response ~###~ {\"cmd\": \"web\", \"params\": {\"query\": \"some query\"}} ~###~.\n6) If you want to execute a command without any response, return only the JSON object.\n7) Responses from commands will be returned in the \"result\" key.\n8) Commands are listed one command per line and each command is described with syntax: \"<name>\": <action>, params: <params>\n9) Always use the correct command name, e.g., if the command name is \"sys_exec\", then use \"sys_exec\" and don't use other names, like \"run\" or something.\n10) With these commands, you are allowed to run external commands and apps in the user's system (environment).\n11) Always use the defined syntax to prevent errors.\n12) Always choose the most appropriate command from the list to perform the task, based on the description of the action performed by a given command.\n13) Reply to the user in the language in which they started the conversation with you.\n14) Use ONLY params described in the command definition, do NOT use any additional params not described on the list.\n15) ALWAYS remember that any text content must appear at the beginning of your response and commands must only be included at the end.\n16) Every command parameter must be placed in one line, so when you generate code you must put all of the code in one line.\n17) Run the commands immediately.\n\nCommands list:", "syntax": ["\"get_ctx_list_in_date_range\": use to get the list of context history (previous conversations between you and me), with corresponding IDs, using the special date-range query syntax: \"@date(YYYY-MM-DD)\" for single day, \"@date(YYYY-MM-DD,)\" for date from, \"@date(,YYYY-MM-DD)\" for date to, and \"@date(YYYY-MM-DD,YYYY-MM-DD)\" for date range from-to, params: \"range_query\"", "\"get_ctx_content_by_id\": use to get summarized content of context with defined ID, prepare summary query to ask another model to summarize the content, starting from e.g. \"You are an expert in content summarization. Summarize our previous discussion answering the query: (query)\", params: \"id\", \"summary_query\"", "\"count_ctx_in_date\": use to count items of context history (previous conversations between you and me) in specified date, by providing year, month, day or a combination of them, params: \"year\", \"month\", \"day\"", "\"get_day_note\": use to get my day notes and plans for a specific date, params: \"year\", \"month\", \"day\"", "\"add_day_note\": use to add day note for specific date, params: \"year\", \"month\", \"day\", \"note\"", "\"update_day_note\": update content of day note for specific date, params: \"year\", \"month\", \"day\", \"content\"", "\"remove_day_note\": remove day note for specific date, params: \"year\", \"month\", \"day\""]}, "ctx": null, "stop": false, "internal": false}
Appending input to chat window...
Bridge call...
{'mode': 'chat', 'model': '{"id": "gpt-4-1106-preview", "name": "gpt-4-1106-preview", "mode": "chat,assistant,langchain,llama_index,agent", "langchain": {"provider": "openai", "mode": ["chat"], "args": [{"name": "model_name", "value": "gpt-4-1106-preview", "type": "str"}], "env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}]}, "ctx": 128000, "tokens": 4096, "default": false, "langchain.provider": "openai", "langchain.mode": "chat", "langchain.args": [{"name": "model_name", "value": "gpt-4-1106-preview", "type": "str"}], "langchain.env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}], "llama_index.provider": "openai", "llama_index.mode": "chat", "llama_index.args": [{"name": "model", "value": "gpt-4-1106-preview", "type": "str"}], "llama_index.env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}]}', 'ctx': '{"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}', 'prompt': 'Search the web for the next SpaceX events', 'system_prompt': 'You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.\nIMAGE GENERATION: Whenever I provide a basic idea or concept for an image, such as \'a picture of mountains\', I want you to ALWAYS translate it into English and expand and elaborate on this idea. Use your  knowledge and creativity to add details that would make the image more vivid and interesting. This could include specifying the time of day, weather conditions, surrounding environment, and any additional elements that could enhance the scene. Your goal is to create a detailed and descriptive prompt that provides DALL-E  with enough information to generate a rich and visually appealing image. Remember to maintain the original  intent of my request while enriching the description with your imaginative details. HOW TO START IMAGE GENERATION: to start image generation return to me prepared prompt in JSON format, all in one line,  using following syntax: ~###~{"cmd": "image", "params": {"query": "your query here"}}~###~. Use ONLY this syntax and remember to surround JSON string with ~###~. DO NOT use any other syntax. Use English in the generated JSON command, but conduct all the remaining parts of the discussion with me in the language in which I am speaking to you. The image will be generated on my machine  immediately after the command is issued, allowing us to discuss the photo once it has been created.  Please engage with me about the photo itself, not only by giving the generate command. \nADDITIONAL CONTEXT: I will provide you with additional data about my question. When it is provided, then use this data as your additional knowledge and use it in your response. Additional context will be prefixed with an "Additional data:" prefix. You can also provide a command to query my knowledge database anytime you need any additional data - to do this, return to me the prepared prompt in JSON format, all in one line, using the following syntax: ~###~{"cmd": "get_knowledge", "params": {"question": "simple and concrete question here"}}~###~. Use ONLY this syntax and remember to surround the JSON string with ~###~. DO NOT use any other syntax. When making query use language that I spoke to you. RUNNING COMMANDS:\nYou can execute commands and also use them to run commands in the user\'s environment.\n\nImportant rules:\n1) The list of available commands is defined below.\n2) To execute a defined command, return a JSON object with the "cmd" key and the command name as its value.\n3) Always use the syntax defined in the command definition and the correct command name.\n4) Put command parameters in the "params" key. Example: {"cmd": "web", "params": {"query": "some query"}}. Use ONLY this syntax. DO NOT use any other syntax.\n5) Append the JSON object to the response at the end and around it with the `~###~` characters. Example: text response ~###~ {"cmd": "web", "params": {"query": "some query"}} ~###~.\n6) If you want to execute a command without any response, return only the JSON object.\n7) Responses from commands will be returned in the "result" key.\n8) Commands are listed one command per line and each command is described with syntax: "<name>": <action>, params: <params>\n9) Always use the correct command name, e.g., if the command name is "sys_exec", then use "sys_exec" and don\'t use other names, like "run" or something.\n10) With these commands, you are allowed to run external commands and apps in the user\'s system (environment).\n11) Always use the defined syntax to prevent errors.\n12) Always choose the most appropriate command from the list to perform the task, based on the description of the action performed by a given command.\n13) Reply to the user in the language in which they started the conversation with you.\n14) Use ONLY params described in the command definition, do NOT use any additional params not described on the list.\n15) ALWAYS remember that any text content must appear at the beginning of your response and commands must only be included at the end.\n16) Every command parameter must be placed in one line, so when you generate code you must put all of the code in one line.\n17) Run the commands immediately.\n\nCommands list:\n"get_ctx_list_in_date_range": use to get the list of context history (previous conversations between you and me), with corresponding IDs, using the special date-range query syntax: "@date(YYYY-MM-DD)" for single day, "@date(YYYY-MM-DD,)" for date from, "@date(,YYYY-MM-DD)" for date to, and "@date(YYYY-MM-DD,YYYY-MM-DD)" for date range from-to, params: "range_query"\n"get_ctx_content_by_id": use to get summarized content of context with defined ID, prepare summary query to ask another model to summarize the content, starting from e.g. "You are an expert in content summarization. Summarize our previous discussion answering the query: (query)", params: "id", "summary_query"\n"count_ctx_in_date": use to count items of context history (previous conversations between you and me) in specified date, by providing year, month, day or a combination of them, params: "year", "month", "day"\n"get_day_note": use to get my day notes and plans for a specific date, params: "year", "month", "day"\n"add_day_note": use to add day note for specific date, params: "year", "month", "day", "note"\n"update_day_note": update content of day note for specific date, params: "year", "month", "day", "content"\n"remove_day_note": remove day note for specific date, params: "year", "month", "day"', 'system_prompt_raw': 'You are a helpful assistant.', 'stream': 'True', 'attachments': '{}', 'assistant_id': 'None', 'idx': 'base', 'idx_raw': 'False', 'external_functions': 'None', 'tools_outputs': '[]'}
Dispatch event begin: mode.before
EVENT BEFORE: {"name": "mode.before", "data": {"value": "chat", "prompt": "Search the web for the next SpaceX events"}, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}, "stop": false, "internal": false}
Apply [mode.before] to plugin: real_time
Apply [mode.before] to plugin: audio_openai_whisper
Apply [mode.before] to plugin: cmd_web_google
Apply [mode.before] to plugin: cmd_files
Apply [mode.before] to plugin: cmd_code_interpreter
Apply [mode.before] to plugin: cmd_custom
Apply [mode.before] to plugin: cmd_history
Apply [mode.before] to plugin: openai_dalle
Apply [mode.before] to plugin: openai_vision
Apply [mode.before] to plugin: idx_llama_index
Apply [mode.before] to plugin: crontab
Dispatch event end: mode.before
EVENT AFTER: {"name": "mode.before", "data": {"value": "chat", "prompt": "Search the web for the next SpaceX events"}, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}, "stop": false, "internal": false}
Bridge call (after inline)...
{'mode': 'chat', 'model': '{"id": "gpt-4-1106-preview", "name": "gpt-4-1106-preview", "mode": "chat,assistant,langchain,llama_index,agent", "langchain": {"provider": "openai", "mode": ["chat"], "args": [{"name": "model_name", "value": "gpt-4-1106-preview", "type": "str"}], "env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}]}, "ctx": 128000, "tokens": 4096, "default": false, "langchain.provider": "openai", "langchain.mode": "chat", "langchain.args": [{"name": "model_name", "value": "gpt-4-1106-preview", "type": "str"}], "langchain.env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}], "llama_index.provider": "openai", "llama_index.mode": "chat", "llama_index.args": [{"name": "model", "value": "gpt-4-1106-preview", "type": "str"}], "llama_index.env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}]}', 'ctx': '{"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": null, "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352328, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": true, "internal": false}', 'prompt': 'Search the web for the next SpaceX events', 'system_prompt': 'You are a helpful assistant. Current time is Monday, 2024-02-19 15:18:48.\nIMAGE GENERATION: Whenever I provide a basic idea or concept for an image, such as \'a picture of mountains\', I want you to ALWAYS translate it into English and expand and elaborate on this idea. Use your  knowledge and creativity to add details that would make the image more vivid and interesting. This could include specifying the time of day, weather conditions, surrounding environment, and any additional elements that could enhance the scene. Your goal is to create a detailed and descriptive prompt that provides DALL-E  with enough information to generate a rich and visually appealing image. Remember to maintain the original  intent of my request while enriching the description with your imaginative details. HOW TO START IMAGE GENERATION: to start image generation return to me prepared prompt in JSON format, all in one line,  using following syntax: ~###~{"cmd": "image", "params": {"query": "your query here"}}~###~. Use ONLY this syntax and remember to surround JSON string with ~###~. DO NOT use any other syntax. Use English in the generated JSON command, but conduct all the remaining parts of the discussion with me in the language in which I am speaking to you. The image will be generated on my machine  immediately after the command is issued, allowing us to discuss the photo once it has been created.  Please engage with me about the photo itself, not only by giving the generate command. \nADDITIONAL CONTEXT: I will provide you with additional data about my question. When it is provided, then use this data as your additional knowledge and use it in your response. Additional context will be prefixed with an "Additional data:" prefix. You can also provide a command to query my knowledge database anytime you need any additional data - to do this, return to me the prepared prompt in JSON format, all in one line, using the following syntax: ~###~{"cmd": "get_knowledge", "params": {"question": "simple and concrete question here"}}~###~. Use ONLY this syntax and remember to surround the JSON string with ~###~. DO NOT use any other syntax. When making query use language that I spoke to you. RUNNING COMMANDS:\nYou can execute commands and also use them to run commands in the user\'s environment.\n\nImportant rules:\n1) The list of available commands is defined below.\n2) To execute a defined command, return a JSON object with the "cmd" key and the command name as its value.\n3) Always use the syntax defined in the command definition and the correct command name.\n4) Put command parameters in the "params" key. Example: {"cmd": "web", "params": {"query": "some query"}}. Use ONLY this syntax. DO NOT use any other syntax.\n5) Append the JSON object to the response at the end and around it with the `~###~` characters. Example: text response ~###~ {"cmd": "web", "params": {"query": "some query"}} ~###~.\n6) If you want to execute a command without any response, return only the JSON object.\n7) Responses from commands will be returned in the "result" key.\n8) Commands are listed one command per line and each command is described with syntax: "<name>": <action>, params: <params>\n9) Always use the correct command name, e.g., if the command name is "sys_exec", then use "sys_exec" and don\'t use other names, like "run" or something.\n10) With these commands, you are allowed to run external commands and apps in the user\'s system (environment).\n11) Always use the defined syntax to prevent errors.\n12) Always choose the most appropriate command from the list to perform the task, based on the description of the action performed by a given command.\n13) Reply to the user in the language in which they started the conversation with you.\n14) Use ONLY params described in the command definition, do NOT use any additional params not described on the list.\n15) ALWAYS remember that any text content must appear at the beginning of your response and commands must only be included at the end.\n16) Every command parameter must be placed in one line, so when you generate code you must put all of the code in one line.\n17) Run the commands immediately.\n\nCommands list:\n"get_ctx_list_in_date_range": use to get the list of context history (previous conversations between you and me), with corresponding IDs, using the special date-range query syntax: "@date(YYYY-MM-DD)" for single day, "@date(YYYY-MM-DD,)" for date from, "@date(,YYYY-MM-DD)" for date to, and "@date(YYYY-MM-DD,YYYY-MM-DD)" for date range from-to, params: "range_query"\n"get_ctx_content_by_id": use to get summarized content of context with defined ID, prepare summary query to ask another model to summarize the content, starting from e.g. "You are an expert in content summarization. Summarize our previous discussion answering the query: (query)", params: "id", "summary_query"\n"count_ctx_in_date": use to count items of context history (previous conversations between you and me) in specified date, by providing year, month, day or a combination of them, params: "year", "month", "day"\n"get_day_note": use to get my day notes and plans for a specific date, params: "year", "month", "day"\n"add_day_note": use to add day note for specific date, params: "year", "month", "day", "note"\n"update_day_note": update content of day note for specific date, params: "year", "month", "day", "content"\n"remove_day_note": remove day note for specific date, params: "year", "month", "day"', 'system_prompt_raw': 'You are a helpful assistant.', 'stream': 'True', 'attachments': '{}', 'assistant_id': 'None', 'idx': 'base', 'idx_raw': 'False', 'external_functions': 'None', 'tools_outputs': '[]', 'parent_mode': 'chat'}
Context: OUTPUT: {"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 0, "total_tokens": 0, "extra": null, "current": false, "internal": false}
Reading stream...

** (pygpt:122761): WARNING **: 15:18:49.437: atk-bridge: get_device_events_reply: unknown signature
End of stream.
Dispatch event begin: ctx.after
EVENT BEFORE: {"name": "ctx.after", "data": null, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}, "stop": false, "internal": false}
Apply [ctx.after] to plugin: real_time
Apply [ctx.after] to plugin: audio_openai_whisper
Apply [ctx.after] to plugin: cmd_web_google
Apply [ctx.after] to plugin: cmd_files
Apply [ctx.after] to plugin: cmd_code_interpreter
Apply [ctx.after] to plugin: cmd_custom
Apply [ctx.after] to plugin: cmd_history
Apply [ctx.after] to plugin: openai_dalle
Apply [ctx.after] to plugin: openai_vision
Apply [ctx.after] to plugin: idx_llama_index
Apply [ctx.after] to plugin: crontab
Dispatch event end: ctx.after
EVENT AFTER: {"name": "ctx.after", "data": null, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}, "stop": false, "internal": false}
Appending output to chat window...
Dispatch CMD event begin: cmd.inline
EVENT BEFORE: {"name": "cmd.inline", "data": {"commands": [{"cmd": "web", "params": {"query": "next SpaceX events"}}]}, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [{"cmd": "web", "params": {"query": "next SpaceX events"}}], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}, "stop": false, "internal": false}
Apply [cmd.inline] to plugin: real_time
Apply [cmd.inline] to plugin: audio_openai_whisper
Apply [cmd.inline] to plugin: cmd_web_google
Apply [cmd.inline] to plugin: cmd_files
Apply [cmd.inline] to plugin: cmd_code_interpreter
Apply [cmd.inline] to plugin: cmd_custom
Apply [cmd.inline] to plugin: cmd_history
Apply [cmd.inline] to plugin: openai_dalle
Apply [cmd.inline] to plugin: openai_vision
Apply [cmd.inline] to plugin: idx_llama_index
Apply [cmd.inline] to plugin: crontab
Reply...
CTX REPLY: {"id": 103, "meta_id": null, "external_id": null, "cmds": [{"cmd": "web", "params": {"query": "next SpaceX events"}}], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}
Context: END: {"id": 103, "meta_id": null, "external_id": null, "cmds": [{"cmd": "web", "params": {"query": "next SpaceX events"}}], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}
Bridge quick call...
Dispatch event begin: ctx.end
{'prompt': 'Summarize topic of this conversation in one sentence. Use best keywords to describe it. Summary must be in the same language as the conversation and it will be used for conversation title so it must be EXTREMELY SHORT and concise - use maximum 5 words: \n\nUser: Search the web for the next SpaceX events\nAI Assistant: ~###~{"cmd": "web", "params": {"query": "next SpaceX events"}}~###~', 'system_prompt': 'You are an expert in conversation summarization', 'max_tokens': '500', 'model': '{"id": "gpt-3.5-turbo-1106", "name": "gpt-3.5-turbo-1106", "mode": "chat,assistant,langchain,agent", "langchain": {"provider": "openai", "mode": ["chat"], "args": [{"name": "model_name", "value": "gpt-3.5-turbo-1106", "type": "str"}], "env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}]}, "ctx": 16385, "tokens": 4096, "default": false, "langchain.provider": "openai", "langchain.mode": "chat", "langchain.args": [{"name": "model_name", "value": "gpt-3.5-turbo-1106", "type": "str"}], "langchain.env": [{"name": "OPENAI_API_KEY", "value": "{api_key}"}], "llama_index.provider": null, "llama_index.mode": "", "llama_index.args": [], "llama_index.env": []}'}
EVENT BEFORE: {"name": "ctx.end", "data": null, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [{"cmd": "web", "params": {"query": "next SpaceX events"}}], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}, "stop": false, "internal": false}
Apply [ctx.end] to plugin: real_time
Apply [ctx.end] to plugin: audio_openai_whisper
Apply [ctx.end] to plugin: cmd_web_google
Apply [ctx.end] to plugin: cmd_files
Apply [ctx.end] to plugin: cmd_code_interpreter
Apply [ctx.end] to plugin: cmd_custom
Apply [ctx.end] to plugin: cmd_history
Apply [ctx.end] to plugin: openai_dalle
Apply [ctx.end] to plugin: openai_vision
Apply [ctx.end] to plugin: idx_llama_index
Apply [ctx.end] to plugin: crontab
Dispatch event end: ctx.end
EVENT AFTER: {"name": "ctx.end", "data": null, "ctx": {"id": 103, "meta_id": null, "external_id": null, "cmds": [{"cmd": "web", "params": {"query": "next SpaceX events"}}], "results": [], "urls": [], "images": [], "files": [], "attachments": [], "reply": false, "input": "Search the web for the next SpaceX events", "output": "~###~{\"cmd\": \"web\", \"params\": {\"query\": \"next SpaceX events\"}}~###~", "mode": "chat", "model": "gpt-4-1106-preview", "thread": null, "msg_id": null, "run_id": null, "input_name": "", "output_name": "", "input_timestamp": 1708352328, "output_timestamp": 1708352329, "input_tokens": 1246, "output_tokens": 23, "total_tokens": 1269, "extra": null, "current": false, "internal": false}, "stop": false, "internal": false}
End.
szczyglis-dev commented 4 months ago

Did you enable execution of the commands? I don't see any commands in debug and there is no command "web" in Google plugin, "web_search" is the command.

You must enable "Execute commands" option checkbox to perform web search (and to execute rest of commands from the plugins):

cmd

bradjohnl commented 4 months ago

@szczyglis-dev Thank you. That did seem to unlock the issue, however I get an error:

Command: Google Web Search: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)>

I've done some searching and tried to upgrade the certifi library with pip, however that did not sort any effect. I've had the option "Disable SSL verify" enabled, and I've tried to disable it. Same results.

I will continue to research and try some stuff, but it would be great if you know the solution to this problem if you can share it.

bradjohnl commented 4 months ago

I've tried some things, on my system (Fedora Linux):

System: Fedora Linux.

Troubleshooting Steps Taken:

Verified System Date and Time: Checked and confirmed that the system's date and time are correctly set.

Ensured Operating System is Up-to-Date: Updated the Fedora Linux operating system to ensure all system components, including SSL/TLS libraries, are current.

Updated CA Certificates: Ran sudo dnf install ca-certificates to ensure the latest CA certificates are installed on the system. Used sudo update-ca-trust to refresh the system's CA trust store.

Set Python to Use System Certificates: Configured Python to use system SSL certificates by setting the REQUESTS_CA_BUNDLE environment variable to point to Fedora's certificate bundle (/etc/ssl/certs/ca-bundle.crt).

Updated Python's certifi Package: Upgraded the certifi package in Python using pip install --upgrade certifi to ensure Python has the latest set of certificates for SSL verification.

Checked Python's SSL Configuration: Examined Python’s OpenSSL version using ssl.OPENSSL_VERSION in the Python interpreter to understand the SSL configuration Python is using. It seems I am using a relatively recent version: 3.1.1 (May 2023)

Inspected Remote Server's Certificates: Used openssl s_client -showcerts -connect google.com:443 to inspect the SSL certificate chain of the remote server for any issues or missing certificates. I can confirm that the SSL handshake is successful: --- SSL handshake has read 6813 bytes and written 394 bytes Verification: OK ---

Despite these steps, the issue persists.

I have to say that on another system using Pop!_OS 22.04 I am not facing the issue.

I am using the precompiled binary in both systems

bradjohnl commented 4 months ago

Based on this I think the issue can be closed as it's not related to the application itself, but rather on one specific system.

@szczyglis-dev your first comment addressed what needed to be done, but this other issue on the SSL side, looks like it's not dependent on the application

bradjohnl commented 4 months ago

Solved with SSL_CERT_DIR=/etc/ssl/certs ./pygpt

szczyglis-dev commented 4 months ago

Good to hear that everything is working now ;)

Btw, additionally, in the newest version 2.0.156, I updated certifi in compiled versions to 2024.2.2 and expanded the Disable SSL verify option to also cover connections to the Google search engine (not just the search results as before).