openchatai / OpenChat

LLMs custom-chatbots console ⚡
https://open.cx
MIT License
5.19k stars 641 forks source link

wrong output of \n #162

Open Z-ZHHH opened 1 year ago

Z-ZHHH commented 1 year ago

I run openchat on Webshop task. And the model output the wrong \n like \n, \\n, \\n. For example

'history': [{'observation': "WebShop [SEP] Instruction: [SEP] i'm looking for a topper for a birthday cake, and price lower than 30.00 dollars [SEP] Search", 'available_actions': {'has_search_bar': True, 'clickables': ['search']}, 'response': "{'id': 'chatcmpl-TxbAh6yUHXRtFoGYivTZQ9', 'object': 'chat.completion', 'created': 1695293501, 'model': 'openchat_v3.2_super', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': 'Thought:\\nI think I should use the search bar to look for the product I need.\\n\\nAction:\\nsearch[topper for birthday cake price < 30.00]\\n'}, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 2279, 'total_tokens': 2325, 'completion_tokens': 46}}", 'action': '\\n', 'reward': 0, 'done': False}, 
{'observation': "WebShop [SEP] Instruction: [SEP] i'm looking for a topper for a birthday cake, and price lower than 30.00 dollars [SEP] Search", 'available_actions': {'has_search_bar': True, 'clickables': ['search']}, 'response': '{\'id\': \'chatcmpl-5tfxH6SqPyRDs8s9kexBpJ\', \'object\': \'chat.completion\', \'created\': 1695293524, \'model\': \'openchat_v3.2_super\', \'choices\': [{\'index\': 0, \'message\': {\'role\': \'assistant\', \'content\': "{\'id\': \'chatcmpl-TxbAh6yUHXRtFoGYivTZQ9\', \'object\': \'chat.completion\', \'created\': 1695293501,\'model\': \'openchat_v3.2_super\', \'choices\': [{\'index\': 0,\'message\': {\'role\': \'assistant\', \'content\': \'Thought:\\\\nI think I should use the search bar to look for the product I need.\\\\n\\\\nAction:\\\\nsearch[topper for birthday cake price < 30.00]\\\\n\'},

Beacuase of the problem, all tasks in Webshop failed, and the final result is zero. Could anyone help with that?

codebanesr commented 1 year ago

I want to make sure I give you the right information. Could you explain a bit more about what you need help with? Some additional details would help me better understand the question

Z-ZHHH commented 1 year ago

I want to reproduce the experiments of Openchat v3.2 super on Webshop (78.6% success rate). I installed the Webshop and Openchat environment and they work well. However, the success rate is zero on Webshop, I check the intermidiate response of Openchat and I found it ouput the wrong format of \n like Action: \\n search[ipad], but the correct format is Action: \n search[ipad].