When I use interpreter.chat(stream=True), in what scenarios will type return 'image'? - Githubissues

OpenInterpreter / open-interpreter

A natural language interface for computers

http://openinterpreter.com/

GNU Affero General Public License v3.0

55.49k stars 4.83k forks source link

When I use interpreter.chat(stream=True), in what scenarios will type return 'image'? #1301

Open chasing1874 opened 5 months ago

chasing1874 commented 5 months ago

Describe the bug

When I use interpreter.chat(stream=True), in what scenarios will type return 'image'?

WeChat81712e215a2648b66051feea8cb3e107

When I try to use it in version 0.1.18, it returns image, but version 0.2.5 does not like this： v0.2.5

v0.1.18

v0.1.18 return the LMC msg with type = image， v0.2.5 type=console

I have set interpreter.computer.emit_images=True, but it doesn't seem to make any difference

Reproduce

none

Expected behavior

I wanna know in what case will type return 'image'?

Screenshots

No response

Open Interpreter version

0.2.5

Python version

3.10

Operating System name and version

macos

Additional context

No response

chasing1874 commented 5 months ago

Or when OI generates a chart using MatPlotLib I'm trying to find a way of saving it as a .png and returning the image path in the message thread so that I can display it in a tag on the frontend. Ideally it would be returned like so: {"role": "assistant", "type": "image", "content": image_path} How can I acheive this? Do I need to extend the Interpreter class with an extra method or can i achieve this through system prompt?