Ascii error when running messenger integration

[ghost]

Everytime i run python run.py --config-path ../../tasks/chatbot/config.yml I get Please install 3rd-party dependencies by running: pip install joblib websocket-client sh and then when i run pip install joblib websocket-client sh I get Requirement already satisfied: joblib in f:\achilliosomega\ai_framework\conv_learning_ai\parlai\.parlai\lib\site-packages\joblib-0.14.1-py3.6.egg (0.14.1) Requirement already satisfied: websocket-client in f:\achilliosomega\ai_framework\conv_learning_ai\parlai\.parlai\lib\site-packages\websocket_client-0.56.0-py3.6.egg (0.56.0) Requirement already satisfied: sh in f:\achilliosomega\ai_framework\conv_learning_ai\parlai\.parlai\lib\site-packages\sh-1.12.14-py3.6.egg (1.12.14) Requirement already satisfied: six in f:\achilliosomega\ai_framework\conv_learning_ai\parlai\.parlai\lib\site-packages\six-1.14.0-py3.6.egg (from websocket-client) (1.14.0)

[ghost]

Further Diagnosing looks like this is due to it being installed on windows

[ghost]

To combat this. I have installed pbs and put the workaround in:

try: import sh except ImportError:

fallback: emulate the sh API with pbs

import pbs
class Sh(object):
    def __getattr__(self, attr):
        return pbs.Command(attr)
sh = Sh()

[ghost]

Now though I have been hit with a

Setting up Messenger webhook... Heroku: Collecting files... Traceback (most recent call last): File "run.py", line 47, in run(opt) File "run.py", line 32, in run manager = MessengerManager(opt) File "[REDACTED]\parlai\parlai\chat_service\services\messenger\messenger_manager.py", line 43, in init self._complete_setup() File "[REDACTED]\parlai\parlai\chat_service\services\messenger\messenger_manager.py", line 61, in _complete_setup self.setup_server() File "[REDACTED]\parlai\parlai\chat_service\services\messenger\messenger_manager.py", line 199, in setup_server self.server_task_name, local=self.opt['local'] File "[REDACTED]\parlai\parlai\chat_service\core\server_utils.py", line 297, in setup_server return setup_heroku_server(task_name) File "[REDACTED]\parlai\parlai\chat_service\core\server_utils.py", line 77, in setup_heroku_server heroku_url, os_name, bit_architecture File "[REDACTED]\ParlAI.parlai\lib\site-packages\pbs.py", line 454, in call cwd=call_args["cwd"], stdin=stdin, stdout=stdout, stderr=stderr) File "C:\Users\admin\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 729, in init restore_signals, start_new_session) File "C:\Users\admin\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 1017, in _execute_child startupinfo) FileNotFoundError: [WinError 2] The system cannot find the file specified

[stephenroller]

Thanks for reporting. Unfortunately, we don't support windows right now. If you manage a fix, I'd be very receptive to a pull request.

[ghost]

@stephenroller Hi i am now running on a linux machine with this. Got alot further this time I now get this though

Setting up Messenger webhook...

Heroku: Collecting files... Heroku: Starting server... Traceback (most recent call last): File "/root/ParlAI/parlai/chat_service/core/server_utils.py", line 127, in setup_heroku_server stderr=subprocess.STDOUT, File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/root/ParlAI/parlai/chat_service/core/heroku-cli-v6.99.0-ec9edad-linux-x64/bin/heroku', 'create', 'root-parlai-messenger-chatbot']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "run.py", line 47, in run(opt) File "run.py", line 32, in run manager = MessengerManager(opt) File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 43, in init self._complete_setup() File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 61, in _complete_setup self.setup_server() File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 199, in setup_server self.server_task_name, local=self.opt['local'] File "/root/ParlAI/parlai/chat_service/core/server_utils.py", line 288, in setup_server return setup_heroku_server(task_name) File "/root/ParlAI/parlai/chat_service/core/server_utils.py", line 170, in setup_heroku_server print(error_text) UnicodeEncodeError: 'ascii' codec can't encode character '\u25b8' in position 45: ordinal not in range(128)

[stephenroller]

Assigning to @JackUrb

[ghost]

thanks

[JackUrb]

Hm it seems the error text isn't being properly displayed, which makes it hard to tell what's specifically wrong in your system. The following line may be the problem: https://github.com/facebookresearch/ParlAI/blob/72c304fa7cac16ed19d8bc75a017f17c8073dd2f/parlai/chat_service/core/server_utils.py#L130 I think the proper decode call should read error_text = e.output.decode(), but I'm actually not sure the e.output string needs to be decoded in the first place. (The same bug exists on L150 too).

To be sure you can try adding print(type(e.output), e.output) right before the error_text = line, and if that's printing <class 'str'> <your error> then the error_text = ... calls should be replaced with error_text = e.output. If it's printing <class 'bytes'> <your error> then something is wrong with the decoding.

[ghost]

<class 'bytes'> b'Creating root-parlai-messenger-achillio... !\n \xe2\x96\xb8 Name root-parlai-messenger-achillio is already taken\n'

[ghost]

Delelted the old app which was in there. it has gone further. I get this though

Local: Setting up WebSocket... done with websocket [ warning: overriding opt['subtasks'] to ['dialog', 'satisfaction'] (previously: ['dialog', 'feedback', 'satisfaction'] )] [ warning: overriding opt['interactive_task'] to True (previously: None )] [ warning: overriding opt['prev_response_filter'] to True (previously: False )] [ warning: overriding opt['partial_load'] to True (previously: False )] [ warning: overriding opt['eval_candidates'] to fixed (previously: inline )] [ warning: overriding opt['encode_candidate_vecs'] to True (previously: False )] [ warning: overriding opt['fixed_candidates_path'] to os.path.join(opt['datapath'], 'self_feeding/convai2_cands.txt') (previously: None )] /root/ParlAI/projects/self_feeding/self_feeding_agent.py:293: UserWarning: Old model: overriding add_double_person_tokens to True. warn_once('Old model: overriding add_double_person_tokens to True.') /root/ParlAI/projects/self_feeding/self_feeding_agent.py:252: UserWarning: [ Option --prev-response-filter=True can only be used when --interactive=True. Overriding it to False. ] warn_once(msg) Traceback (most recent call last): File "run.py", line 47, in run(opt) File "run.py", line 32, in run manager = MessengerManager(opt) File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 43, in init self._complete_setup() File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 65, in _complete_setup self._load_model() File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 72, in _load_model self.runner_opt['shared_bot_params'] = create_agent(self.runner_opt).share() File "/root/ParlAI/parlai/core/agents.py", line 388, in create_agent model = create_agent_from_opt_file(opt) File "/root/ParlAI/parlai/core/agents.py", line 343, in create_agent_from_opt_file return model_class(new_opt) File "/root/ParlAI/projects/self_feeding/self_feeding_agent.py", line 253, in init self.opt['prev_response_filter'] = False AttributeError: 'SelfFeedingAgent' object has no attribute 'opt'

My config.yml is this

tasks: default: onboard_world: MessengerBotChatOnboardWorld task_world: MessengerBotChatTaskWorld timeout: 1800 agents_required: 1 task_name: achilliosomega-heroku world_module: parlai.chat_service.tasks.chatbot.worlds overworld: MessengerOverworld max_workers: 30 opt: # Additional model opts go here model_file: zoo:self_feeding/hh131k_hb60k_fb60k_st1k/model override: subtasks: ['dialog', 'satisfaction'] interactive_task: True prev_response_filter: True person_tokens: False # SelfFeedingAgent adds person_tokens on its own partial_load: True history_size: 2 eval_candidates: 'fixed' encode_candidate_vecs: True fixed_candidates_path: os.path.join(opt['datapath'], 'self_feeding/convai2_cands.txt') additional_args: page_id: 108160617431926 # configure your own page

[ghost]

With some adjusting i have managed to change the error message to:

Dictionary: loading dictionary from /root/ParlAI/data/models/self_feeding/hh131k_hb60k_fb60k_st1k/model.dict [ num words = 23617 ] Traceback (most recent call last): File "run.py", line 47, in run(opt) File "run.py", line 32, in run manager = MessengerManager(opt) File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 43, in init self._complete_setup() File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 65, in _complete_setup self._load_model() File "/root/ParlAI/parlai/chat_service/services/messenger/messenger_manager.py", line 72, in _load_model self.runner_opt['shared_bot_params'] = create_agent(self.runner_opt).share() File "/root/ParlAI/parlai/core/agents.py", line 388, in create_agent model = create_agent_from_opt_file(opt) File "/root/ParlAI/parlai/core/agents.py", line 343, in create_agent_from_opt_file return model_class(new_opt) File "/root/ParlAI/projects/self_feeding/self_feeding_agent.py", line 256, in init super().init(opt, shared) File "/root/ParlAI/parlai/agents/transformer/transformer.py", line 206, in init super().init(opt, shared) File "/root/ParlAI/parlai/core/torch_ranker_agent.py", line 161, in init super().init(opt, shared) File "/root/ParlAI/parlai/core/torch_agent.py", line 723, in init self.set_interactive_mode(opt['interactive_mode'], shared) KeyError: 'interactive_mode'

[ghost]

@JackUrb Are you able to assist? Im guessing there is an issue with my config. When I asked previously I was told it was okay

[JackUrb]

You may be able to solve this by adding interactive: true to the config.yml file.

[ghost]

Ive added it, i get the same message. Do i put it under override?

[ghost]

I have put it in the opt section. If i put it in override i get the same message anyway

[JackUrb]

Sorry - it's looking for interactive_mode: true

[ghost]

This seems to be all working now. Ill close once it has completed. If not ill follow up. Thank you tho

[ghost]

Traceback (most recent call last): File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run result = self.fn(*self.args, **self.kwargs) File "/root/ParlAI/parlai/chat_service/core/manager_utils.py", line 291, in _world_fn return self._run_world(task, world_name, agents) File "/root/ParlAI/parlai/chat_service/core/manager_utils.py", line 260, in _run_world world = world_generator(self.opt, agents) File "/root/ParlAI/parlai/chat_service/tasks/chatbot/worlds.py", line 43, in generate_world if opt['model'] is None and opt['model_file'] is None: KeyError: 'model' @JackUrb From this im guessing im missing a model atribute opt['model']

What model does the self-feeding bot use?

[danielriddell21]

Was there any update to this?

[JackUrb]

Details on using the self feeding model are in its project dir. Please refer to the docs to familiarize yourself with the rest of the standard ParlAI options (such as loading models with -mf).

[ghost]

Looking at the documentation i have managed to fix the model issue. when talking to it i get this error:

World default had error RuntimeError('Message already contains key id. If this was intentional, please use the function force_set(key, value).',) Traceback (most recent call last): File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run result = self.fn(*self.args, self.kwargs) File "/root/ParlAI/parlai/chat_service/core/manager_utils.py", line 291, in _world_fn return self._run_world(task, world_name, agents) File "/root/ParlAI/parlai/chat_service/core/manager_utils.py", line 264, in _run_world ret_val = world.parley() File "/root/ParlAI/parlai/chat_service/tasks/chatbot/worlds.py", line 77, in parley response['id'] = '' File "/root/ParlAI/parlai/core/message.py", line 26, in setitem 'please use the function force_set(key, value).'.format(key) RuntimeError: Message already contains key id. If this was intentional, please use the function force_set(key, value).**

[ghost]

Found this: https://github.com/facebookresearch/ParlAI/issues/2277

[stephenroller]

@klshuster This line appears to be broken, but the context is lost on me:

https://github.com/facebookresearch/ParlAI/blob/bfb4ef291c2d1ea84691efee55a1e4c0e1b8ce36/parlai/chat_service/tasks/chatbot/worlds.py#L77

[JackUrb]

Ah we set the id to nothing in messenger before the human agent observes it for visual reasons. Updating to force_set may fix this.

[ghost]

Is this something I can do? What do i need to do with the force_set?

[JackUrb]

Replace the noted line above with response.force_set('id', '').

[ghost]

thank you!

[ghost]

Last thing before i close this ticket is how when sending the first message can i make it say 'hello' and the users first name. Looking at the documentation doesnt seem to help. maybe im understanding it wrong

[JackUrb]

This would require a change to the worlds.py file, and would also require you being able to retrieve the name of the user. We don't manually parse that out for you in the chat_service architecture, so you'd need to add your own handlers to request the user information via page scoped ID. Overall this would actually be a fairly involved change.

Alternately, you can ask the user how they'd like to be called, and then store that name in some database somewhere for PSID -> name and use that name in the future.

[github-actions[bot]]

This issue has not had activity in 30 days. Marking as stale.

[aslucc]

<class 'bytes'> b'Creating root-parlai-messenger-achillio... !\n \xe2\x96\xb8 Name root-parlai-messenger-achillio is already taken\n'

Hello, I have the same issue, how did you delete the old app?

[JackUrb]

You can do this from the heroku webpage

[aslucc]

Thanks, I guessed so but there are no apps in my heroku profile. Also, if try to delete the app from terminal with heroku apps:destroy root-parlai-messenger-chatbot I get You do not have access to the app root-parlai-messenger-chatbot. I feel like I'm missing something, like root-parlai-messenger-chatbot is a app created by you so I can't name another one the same way but I don't know where to type the name I would like to assign to mine.

[JackUrb]

Oh interesting, the problem is the name itself - the system is supposed to generate a uuid for this reason, but I suppose it's not in your case. You may need to tinker with the server_utils.py file to use a new name