composite_demo 代码解释器 无法生成 图片

[Aike505]

System Info / 系統信息

Python3.10

Who can help? / 谁可以帮助到您？

@abmfy

Information / 问题信息

• [X] The official example scripts / 官方的示例脚本
• [ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

1、修改demo_ci.py， 添加打印如下中间步骤

    def execute(self, code):
        self.kernel.execute(code)
        print("******** execute start **********")
        print("execute->code", code)
        print("self.kernel.execute(code):",self.kernel.execute(code))
        try:
            shell_msg = self.kernel.get_shell_msg(timeout=30)
            print("execute->shell_msg: ", shell_msg)
            io_msg_content = self.kernel.get_iopub_msg(timeout=30)['content']
            print("execute->io_msg_content: ", io_msg_content)
            while True:
                msg_out = io_msg_content
                ### Poll the message
                try:
                    io_msg_content = self.kernel.get_iopub_msg(timeout=30)['content']
                    print("execute->io_msg_content->tyr->while->try: ", io_msg_content)
                    if 'execution_state' in io_msg_content and io_msg_content['execution_state'] == 'idle':
                        break
                except queue.Empty:
                    print("execute->io_msg_content->try->while->except: ", queue.Empty)
                    break
            print("execute->try=shell_msg:",shell_msg)
            print("execute->try=msg_out:",msg_out)
            return shell_msg, msg_out
        except Exception as e:
            print(e)
            print("execute->except: ", e)
            return None

3. 当不要求代码解释器处理图片时，代码解释器正常工作 output {'data': {'text/plain': '33'}, 'metadata': {}, 'execution_count': 4}

=================================
******** execute start **********
execute->code # Calculating the sum of 11 and 22
result = 11 + 22
result

self.kernel.execute(code): 7ca77dc7-acdc3edc036bc1fd74786f70_830_5
execute->shell_msg:  {'header': {'msg_id': 'ea78b957-7c6a275576d5eb363b92df51_945_14', 'msg_type': 'execute_reply', 'username': 'username', 'session': 'ea78b957-7c6a275576d5eb363b92df51', 'date': datetime.datetime(2024, 4, 28, 12, 45, 5, 615395, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': 'ea78b957-7c6a275576d5eb363b92df51_945_14', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '7ca77dc7-acdc3edc036bc1fd74786f70_830_2', 'msg_type': 'execute_request', 'username': 'username', 'session': '7ca77dc7-acdc3edc036bc1fd74786f70', 'date': datetime.datetime(2024, 4, 28, 12, 45, 5, 606512, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {'started': '2024-04-28T12:45:05.609277Z', 'dependencies_met': True, 'engine': 'd6fd8465-6102-47bf-9f08-7b5467d42918', 'status': 'ok'}, 'content': {'status': 'ok', 'execution_count': 3, 'user_expressions': {}, 'payload': []}, 'buffers': []}
execute->io_msg_content:  {'execution_state': 'busy'}
execute->io_msg_content->tyr->while->try:  {'code': '# Calculating the sum of 11 and 22\nresult = 11 + 22\nresult\n', 'execution_count': 4}
execute->io_msg_content->tyr->while->try:  {'data': {'text/plain': '33'}, 'metadata': {}, 'execution_count': 4}
execute->io_msg_content->tyr->while->try:  {'execution_state': 'idle'}
execute->try=shell_msg: {'header': {'msg_id': 'ea78b957-7c6a275576d5eb363b92df51_945_14', 'msg_type': 'execute_reply', 'username': 'username', 'session': 'ea78b957-7c6a275576d5eb363b92df51', 'date': datetime.datetime(2024, 4, 28, 12, 45, 5, 615395, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': 'ea78b957-7c6a275576d5eb363b92df51_945_14', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '7ca77dc7-acdc3edc036bc1fd74786f70_830_2', 'msg_type': 'execute_request', 'username': 'username', 'session': '7ca77dc7-acdc3edc036bc1fd74786f70', 'date': datetime.datetime(2024, 4, 28, 12, 45, 5, 606512, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {'started': '2024-04-28T12:45:05.609277Z', 'dependencies_met': True, 'engine': 'd6fd8465-6102-47bf-9f08-7b5467d42918', 'status': 'ok'}, 'content': {'status': 'ok', 'execution_count': 3, 'user_expressions': {}, 'payload': []}, 'buffers': []}
execute->try=msg_out: {'data': {'text/plain': '33'}, 'metadata': {}, 'execution_count': 4}
~~~~~~~~~~
msg {'header': {'msg_id': 'ea78b957-7c6a275576d5eb363b92df51_945_14', 'msg_type': 'execute_reply', 'username': 'username', 'session': 'ea78b957-7c6a275576d5eb363b92df51', 'date': datetime.datetime(2024, 4, 28, 12, 45, 5, 615395, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': 'ea78b957-7c6a275576d5eb363b92df51_945_14', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '7ca77dc7-acdc3edc036bc1fd74786f70_830_2', 'msg_type': 'execute_request', 'username': 'username', 'session': '7ca77dc7-acdc3edc036bc1fd74786f70', 'date': datetime.datetime(2024, 4, 28, 12, 45, 5, 606512, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {'started': '2024-04-28T12:45:05.609277Z', 'dependencies_met': True, 'engine': 'd6fd8465-6102-47bf-9f08-7b5467d42918', 'status': 'ok'}, 'content': {'status': 'ok', 'execution_count': 3, 'user_expressions': {}, 'payload': []}, 'buffers': []}
output {'data': {'text/plain': '33'}, 'metadata': {}, 'execution_count': 4}
status: ok
~~~~~~~~~
Received: text 33

== Input ==
 33

==History==
 [{'role': 'system', 'content': '你是一位智能AI助手，你叫ChatGLM，你连接着一台电脑，但请注意不能联网。在使用Python解决任务时，你可以运行代码并得到结果，如果运行结果有错误，你需要尽可能对代码进行改进。你可以处理用户上传到电脑上的文件，文件默认存储路径是/。'}, {'role': 'user', 'content': '用python 计算11+22'}, {'role': 'assistant', 'content': '```python\n# Calculating the sum of 11 and 22\nresult = 11 + 22\nresult\n```'}]

3. 当使用代码解释器 输出图片时，大模型未接收到图片。 output {'code': "import matplotlib.pyplot as plt\r\nimport numpy as np\r\n\r\n# 创建一个图形和坐标轴\r\nfig, ax = plt.subplots()\r\n\r\n# 创建一个圆形对象，半径为5\r\ncircle = plt.Circle((0, 0), 5, color='blue', zorder=10)\r\n\r\n# 将圆形添加到坐标轴上\r\nax.add_artist(circle)\r\n\r\n# 设置坐标轴的范围和比例\r\nax.set_xlim(-10, 10)\r\nax.set_ylim(-10, 10)\r\nax.set_aspect('equal')\r\n\r\n# 显示图形\r\nplt.show()\n", 'execution_count': 2}

=================================
******** execute start **********
execute->code import matplotlib.pyplot as plt
import numpy as np

# 创建一个图形和坐标轴
fig, ax = plt.subplots()

# 创建一个圆形对象，半径为5
circle = plt.Circle((0, 0), 5, color='blue', zorder=10)

# 将圆形添加到坐标轴上
ax.add_artist(circle)

# 设置坐标轴的范围和比例
ax.set_xlim(-10, 10)
ax.set_ylim(-10, 10)
ax.set_aspect('equal')

# 显示图形
plt.show()

self.kernel.execute(code): 35fe72b3-09f76574f4dccbbdc50c5d3b_1184_3
execute->shell_msg:  {'header': {'msg_id': '261bdaf7-57cebcd1e127efa5b7845b2e_1229_7', 'msg_type': 'execute_reply', 'username': 'username', 'session': '261bdaf7-57cebcd1e127efa5b7845b2e', 'date': datetime.datetime(2024, 4, 28, 12, 51, 9, 301077, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '261bdaf7-57cebcd1e127efa5b7845b2e_1229_7', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '35fe72b3-09f76574f4dccbbdc50c5d3b_1184_1', 'msg_type': 'execute_request', 'username': 'username', 'session': '35fe72b3-09f76574f4dccbbdc50c5d3b', 'date': datetime.datetime(2024, 4, 28, 12, 51, 7, 912727, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {'started': '2024-04-28T12:51:09.269681Z', 'dependencies_met': True, 'engine': '3c940962-0d8b-41fc-ad35-5f676bc2801a', 'status': 'ok'}, 'content': {'status': 'ok', 'execution_count': 2, 'user_expressions': {}, 'payload': []}, 'buffers': []}
execute->io_msg_content:  {'execution_state': 'busy'}
execute->io_msg_content->tyr->while->try:  {'code': "import matplotlib.pyplot as plt\r\nimport numpy as np\r\n\r\n# 创建一个图形和坐标轴\r\nfig, ax = plt.subplots()\r\n\r\n# 创建一个圆形对象，半径为5\r\ncircle = plt.Circle((0, 0), 5, color='blue', zorder=10)\r\n\r\n# 将圆形添加到坐标轴上\r\nax.add_artist(circle)\r\n\r\n# 设置坐标轴的范围和比例\r\nax.set_xlim(-10, 10)\r\nax.set_ylim(-10, 10)\r\nax.set_aspect('equal')\r\n\r\n# 显示图形\r\nplt.show()\n", 'execution_count': 2}
execute->io_msg_content->tyr->while->try:  {'execution_state': 'idle'}
execute->try=shell_msg: {'header': {'msg_id': '261bdaf7-57cebcd1e127efa5b7845b2e_1229_7', 'msg_type': 'execute_reply', 'username': 'username', 'session': '261bdaf7-57cebcd1e127efa5b7845b2e', 'date': datetime.datetime(2024, 4, 28, 12, 51, 9, 301077, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '261bdaf7-57cebcd1e127efa5b7845b2e_1229_7', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '35fe72b3-09f76574f4dccbbdc50c5d3b_1184_1', 'msg_type': 'execute_request', 'username': 'username', 'session': '35fe72b3-09f76574f4dccbbdc50c5d3b', 'date': datetime.datetime(2024, 4, 28, 12, 51, 7, 912727, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {'started': '2024-04-28T12:51:09.269681Z', 'dependencies_met': True, 'engine': '3c940962-0d8b-41fc-ad35-5f676bc2801a', 'status': 'ok'}, 'content': {'status': 'ok', 'execution_count': 2, 'user_expressions': {}, 'payload': []}, 'buffers': []}
execute->try=msg_out: {'code': "import matplotlib.pyplot as plt\r\nimport numpy as np\r\n\r\n# 创建一个图形和坐标轴\r\nfig, ax = plt.subplots()\r\n\r\n# 创建一个圆形对象，半径为5\r\ncircle = plt.Circle((0, 0), 5, color='blue', zorder=10)\r\n\r\n# 将圆形添加到坐标轴上\r\nax.add_artist(circle)\r\n\r\n# 设置坐标轴的范围和比例\r\nax.set_xlim(-10, 10)\r\nax.set_ylim(-10, 10)\r\nax.set_aspect('equal')\r\n\r\n# 显示图形\r\nplt.show()\n", 'execution_count': 2}
~~~~~~~~~~
msg {'header': {'msg_id': '261bdaf7-57cebcd1e127efa5b7845b2e_1229_7', 'msg_type': 'execute_reply', 'username': 'username', 'session': '261bdaf7-57cebcd1e127efa5b7845b2e', 'date': datetime.datetime(2024, 4, 28, 12, 51, 9, 301077, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '261bdaf7-57cebcd1e127efa5b7845b2e_1229_7', 'msg_type': 'execute_reply', 'parent_header': {'msg_id': '35fe72b3-09f76574f4dccbbdc50c5d3b_1184_1', 'msg_type': 'execute_request', 'username': 'username', 'session': '35fe72b3-09f76574f4dccbbdc50c5d3b', 'date': datetime.datetime(2024, 4, 28, 12, 51, 7, 912727, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {'started': '2024-04-28T12:51:09.269681Z', 'dependencies_met': True, 'engine': '3c940962-0d8b-41fc-ad35-5f676bc2801a', 'status': 'ok'}, 'content': {'status': 'ok', 'execution_count': 2, 'user_expressions': {}, 'payload': []}, 'buffers': []}
output {'code': "import matplotlib.pyplot as plt\r\nimport numpy as np\r\n\r\n# 创建一个图形和坐标轴\r\nfig, ax = plt.subplots()\r\n\r\n# 创建一个圆形对象，半径为5\r\ncircle = plt.Circle((0, 0), 5, color='blue', zorder=10)\r\n\r\n# 将圆形添加到坐标轴上\r\nax.add_artist(circle)\r\n\r\n# 设置坐标轴的范围和比例\r\nax.set_xlim(-10, 10)\r\nax.set_ylim(-10, 10)\r\nax.set_aspect('equal')\r\n\r\n# 显示图形\r\nplt.show()\n", 'execution_count': 2}
status: ok
~~~~~~~~~
Received: None 

== Input ==

==History==
 [{'role': 'system', 'content': '你是一位智能AI助手，你叫ChatGLM，你连接着一台电脑，但请注意不能联网。在使用Python解决任务时，你可以运行代码并得到结果，如果运行结果有错误，你需要尽可能对代码进行改进。你可以处理用户上传到电脑上的文件，文件默认存储路径是/。'}, {'role': 'user', 'content': '画一个圆'}, {'role': 'assistant', 'content': "```python\nimport matplotlib.pyplot as plt\r\nimport numpy as np\r\n\r\n# 创建一个图形和坐标轴\r\nfig, ax = plt.subplots()\r\n\r\n# 创建一个圆形对象，半径为5\r\ncircle = plt.Circle((0, 0), 5, color='blue', zorder=10)\r\n\r\n# 将圆形添加到坐标轴上\r\nax.add_artist(circle)\r\n\r\n# 设置坐标轴的范围和比例\r\nax.set_xlim(-10, 10)\r\nax.set_ylim(-10, 10)\r\nax.set_aspect('equal')\r\n\r\n# 显示图形\r\nplt.show()\n```"}]

Expected behavior / 期待表现

代码解释器调用内核生成的图片能够返回

[sirizhou]

我遇到了同样问题，参考 #459步骤仍然未得到图像。

[aabbccddwasd]

我成功解决了这个问题 在ipython kernel install --name chatglm3-demo --user创建完内核后在.local/share/jupyter/kernels/修改kernel.json,在argv里增加"--matplotlib=inline"这样就可以正常显示图片了