OpenInterpreter / open-interpreter

A natural language interface for computers
http://openinterpreter.com/
GNU Affero General Public License v3.0
50.54k stars 4.4k forks source link

When I use interpreter.chat(stream=True), in what scenarios will type return 'image'? #1301

Open chasing1874 opened 3 weeks ago

chasing1874 commented 3 weeks ago

Describe the bug

When I use interpreter.chat(stream=True), in what scenarios will type return 'image'?

WeChat81712e215a2648b66051feea8cb3e107

When I try to use it in version 0.1.18, it returns image, but version 0.2.5 does not like this: v0.2.5 image

v0.1.18 image

v0.1.18 return the LMC msg with type = image, v0.2.5 type=console

I have set interpreter.computer.emit_images=True, but it doesn't seem to make any difference

Reproduce

none

Expected behavior

I wanna know in what case will type return 'image'?

Screenshots

No response

Open Interpreter version

0.2.5

Python version

3.10

Operating System name and version

macos

Additional context

No response

chasing1874 commented 3 weeks ago

Or when OI generates a chart using MatPlotLib I'm trying to find a way of saving it as a .png and returning the image path in the message thread so that I can display it in a tag on the frontend. Ideally it would be returned like so: {"role": "assistant", "type": "image", "content": image_path} How can I acheive this? Do I need to extend the Interpreter class with an extra method or can i achieve this through system prompt?