mnotgod96 / AppAgent

AppAgent: Multimodal Agents as Smartphone Users, an LLM-based multimodal agent framework designed to operate smartphone apps.
https://appagent-official.github.io/
MIT License
4.97k stars 538 forks source link

ERROR: an exception occurs while parsing the model response: list index out of range #70

Open daimaxiansheng opened 6 months ago

daimaxiansheng commented 6 months ago

Warning! No module named 'sounddevice' Warning! No module named 'matplotlib' Warning! No module named 'keras' Welcome to the deployment phase of AppAgent! Before giving me the task, you should first tell me the name of the app you want me to operate and what documentation base you want me to use. I will try my best to complete the task without your intervention. First, please enter the main interface of the app on your phone and provide the following information.

What is the name of the target app?

华为商城 Warning! No module named 'sounddevice' Warning! No module named 'matplotlib' Warning! No module named 'keras' Documentations generated from human demonstration were found for the app 华为商城. The doc base is selected automatically.

List of devices attached: ['SED0221831016908']

Device selected: SED0221831016908

Screen resolution of SED0221831016908: 1228x2700

Please enter the description of the task you want me to complete in a few sentences:

打开华为商城搜索手表watch4pro加入购物车然后从购物车删除并且关闭华为商城 Round 1

Documentations retrieved for the current interface: Documentation of UI element labeled with the numeric tag '14': This UI element is clickable. The UI element is a button that navigates to the Huawei online store.

Thinking about what to do in the next step...

ERROR: an exception occurs while parsing the model response: list index out of range

Observation:

Task finished unexpectedly

手动模式完成后执行run.py根据要求输入后直接报错了,数组越界,这个该如何解决?

vikingden8 commented 6 months ago

same issue too

daimaxiansheng commented 6 months ago

"choices":[ { "finish_reason":"stop", "message":{ "role":"assistant", "content":[ { "text":"Observation:" } ] } } ] 查了下大模型并没有按照要求返回,仅仅返回了一个Observation,目前来看估计得修改prompt了

jaypatelp001 commented 6 months ago

try to install python pacakages that it indicates, i am also facing same issues after installing libraries manually it works fine.

vikingden8 commented 6 months ago

all the python packages by manual?@

try to install python pacakages that it indicates, i am also facing same issues after installing libraries manually it works fine.

haroqiu commented 6 months ago

Hi, there. This can be solved by simply modify QWEN_MODEL to qwen-vl-plus in config.yaml if you are using Qwen model. The original QWEN_MODEL is qwen-vl-max and I found that qwen-vl-plus is more capable in the context of AppAgent.

daimaxiansheng commented 6 months ago

@Harold-QIU Thank you, it has been confirmed that

Hi, there. This can be solved by simply modify QWEN_MODEL to qwen-vl-plus in config.yaml if you are using Qwen model. The original QWEN_MODEL is qwen-vl-max and I found that qwen-vl-plus is more capable in the context of AppAgent.

CookSleep commented 5 months ago

Hi, there. This can be solved by simply modify QWEN_MODEL to qwen-vl-plus in config.yaml if you are using Qwen model.您好。如果您使用的是 Qwen 模型,只需在 config.yaml 中将 QWEN_MODEL 修改为 qwen-vl-plus 即可解决这个问题。 The original QWEN_MODEL is qwen-vl-max and I found that qwen-vl-plus is more capable in the context of AppAgent.最初的 QWEN_MODELqwen-vl-max ,我发现 qwen-vl-plus 在 AppAgent 中的功能更强。

However, the official description of qwen-vl-max is "Qwen Super Large Scale Visual Language Model. Compared to the plus version, it further enhances visual reasoning capabilities and command compliance abilities, providing higher levels of visual perception and cognition. It offers optimal performance on more complex tasks." ...I can't understand...

zhenchuanpeng commented 5 months ago

The same question

lzr945 commented 2 months ago

请问下,最后是怎么解决了这个问题的,我也同样遇到了,谢谢您!

lzr945 commented 2 months ago

现在使用qwen-vl-plus也还是会报这个错误

haroqiu commented 2 months ago

现在使用qwen-vl-plus也还是会报这个错误

I think you can try the currently latest model every time you meet this issue, since this issue seems strongly correlated with the capability of the model to follow the instruction.

Vermouth-hub commented 1 month ago

same issue!