Handle user location - Githubissues

clalevee commented 7 years ago

I'm trying to handle a location sent by user with FB Messenger (something like pull request #16)

I'm able to intercept and change message in middleware.before (add input.text, update Conversation variable lat/long). In Watson Conversation, message is well received, and an answer is generated and sent back to FB Messenger user. But... FB Messenger user never receive this message. Any idea ?

Code and trace in attached file below

Issue_BotKit_2.pdf

stevenoh93 commented 7 years ago

Thanks for the detailed trace again. It's cool that you're trying to get location with this. Could you show me what your slackController.hears looks like? Are you using interpret or receive? And finally, if you could also show me what message.watsonData looks like in slackController.hears, that would be great.

Thanks, Steven

clalevee commented 7 years ago

I'm using Facebook Messenger and not (yet) Slack. My messenger controller hear is (I didn't modify it, just added console.log).

var bot = controller.spawn();
controller.hears('(.*)', 'message_received', function(bot, message) {
  console.log("--------------------------------");
  console.log("controller.hears");
  console.log(message.watsonData);
  console.log("--------------------------------");
  bot.reply(message, message.watsonData.output.text.join('\n'));
});

module.exports.controller = controller;
module.exports.bot = bot;

When I send location, I can see trace in middleware.after but not in FB controller.hears...

stevenoh93 commented 7 years ago

@clalevee We found a small bug in our package. Could you update your local one and see if you have the same issues?

clalevee commented 7 years ago

Same, not better, with npm botkit-middleware-watson": "^1.3.0"... The strange thing is that I can get "input" Facebook location message, change it into middleware.before, and answer with Watson Conversation : so I see it in middleware.after (see hereunder) but, not in Facebook messenger.

 response after
 { intents: [ { intent: 'store-distance', confidence: 1 } ],
   entities: [],
   input: { text: 'location' },
   output:
    { log_messages: [],
      text: [ 'I understand that you are here: latitude 43.617299753282 & longitude 3.9075415860898' ],
      nodes_visited: [ 'node_6_1483719981993' ] },
   context:
    { conversation_id: 'e07a5810-5988-4892-84e1-1575b9e49757',
      system:
       { dialog_stack: [Object],
         dialog_turn_counter: 6,
         dialog_request_counter: 6,
         _node_output_map: [Object] },
      SQL_Result: '',
      SQL_Request: 'STORE_DISTANCE',
      Launch_Request: '',
      store: 'Polygone',
      lat: 43.617299753282,
      long: 3.9075415860898,
      Reservation_Number: '' } }

stevenoh93 commented 7 years ago

Could you show us your middleware.after code?

clalevee commented 7 years ago

I added a lot of "console.log" to trace, to try to understand.

middleware.after = function(message, conversationResponse, callback) {
    console.log("----------------------------------------");
    console.log("message after ");
    console.log(message);
    console.log("----------------------------------------");
    console.log("response after ");
    console.log(conversationResponse);
    console.log("----------------------------------------");

    if (conversationResponse.context) {
         console.log("conversationResponse.context is not null");

         if (conversationResponse.context.SQL_Request == "QUERY_ORDER" &&
             conversationResponse.context.Launch_Request == "Yes") {
                      console.log("Query RMS Database");
                      console.log("----------------------------------------");
                      ....
                      callback(null, RMS-DB-response);
         }
        else {
                      console.log("Nothing to do, no Query Order. ");
                      console.log("----------------------------------------");
                      callback(null, conversationResponse);
        }
   }
   else {
      console.log("No ConversationResponse.context. ");
      console.log("----------------------------------------");
      callback(null, conversationResponse);
   }
  }

That gives this trace when sending location

 ----------------------------------------
 message after
 { text: 'location',
   user: '1172085072846787',
   channel: '1172085072846787',
   timestamp: 1484735739388,
   seq: 1435,
   mid: 'mid.1484735739388:a1e0438668',
   watsonData: { output: { text: [] } } }
 ----------------------------------------
 response after
 { intents: [ { intent: 'store-distance', confidence: 1 } ],
   entities: [],
   input: { text: 'location' },
   output:
    { log_messages: [],
      text: [ 'I understand that you are here: latitude 43.617369630204 & longitude 3.9076284272982' ],
      nodes_visited: [ 'node_6_1483719981993' ] },
   context:
    { conversation_id: '2fbbbfe5-37ac-4e6b-b601-16679d655d5b',
      system:
       { dialog_stack: [Object],
         dialog_turn_counter: 3,
         dialog_request_counter: 3,
         _node_output_map: [Object] },
      SQL_Result: '',
      SQL_Request: 'STORE_DISTANCE',
      Launch_Request: '',
      store: 'Polygone',
      lat: 43.617369630204,
      long: 3.9076284272982 } }
 ----------------------------------------
 conversationResponse.context is not null
 Nothing to do, no Query Order.
 ----------------------------------------

Message "I understand that you are here: latitude 43.617369630204 & longitude 3.9076284272982" never appears in Messenger.

I also tried to "try.. catch" (required on Bluemix to be sure to catch all errors), but no errors.

clalevee commented 7 years ago

Any update ?

stevenoh93 commented 7 years ago

@clalevee Sorry for the late response.. I've been busy with other projects.. Could you also show me what message.watsonData looks like in your controller.hears function?

clalevee commented 7 years ago

Hi Steven

When I send location message, I get a "Before event", then an "After event" but no Controller.hears. I added console.log in FB controller.hears as bellow

var bot = controller.spawn();
controller.hears('(.*)', 'message_received', function(bot, message) {
  console.log("controller.hears: message.watsonData");
  console.log(message.watsonData);
  bot.reply(message, message.watsonData.output.text.join('\n'));
});

and I got trace bellow (with comments)

——————————————————————————————————————————————
Step1: send message = “how far is Polygone” 
——————————————————————————————————————————————
-> Before event 

2017-02-08T15:57:14.18+0100 [APP/0]      OUT message before
2017-02-08T15:57:14.20+0100 [APP/0]      OUT { text: 'How far is polygone',
2017-02-08T15:57:14.20+0100 [APP/0]      OUT   user: '1172085072846787',
2017-02-08T15:57:14.20+0100 [APP/0]      OUT   channel: '1172085072846787',
2017-02-08T15:57:14.20+0100 [APP/0]      OUT   timestamp: 1486565833753,
2017-02-08T15:57:14.20+0100 [APP/0]      OUT   seq: 1567,
2017-02-08T15:57:14.20+0100 [APP/0]      OUT   mid: 'mid.1486565833753:ca0ac6ac76',
2017-02-08T15:57:14.20+0100 [APP/0]      OUT   attachments: undefined }
2017-02-08T15:57:14.22+0100 [APP/0]      OUT ----------------------------------------
2017-02-08T15:57:14.22+0100 [APP/0]      OUT payload before
2017-02-08T15:57:14.22+0100 [APP/0]      OUT { workspace_id: '934773b3-ed68-43b3-aef2-ad3086ac587a',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT   input: { text: 'How far is polygone' },
2017-02-08T15:57:14.22+0100 [APP/0]      OUT   context:
2017-02-08T15:57:14.22+0100 [APP/0]      OUT    { conversation_id: '2bbecc1a-ce36-4eb1-85ec-d8504ce1547e',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      system:
2017-02-08T15:57:14.22+0100 [APP/0]      OUT       { dialog_stack: [Object],
2017-02-08T15:57:14.22+0100 [APP/0]      OUT         dialog_turn_counter: 7,
2017-02-08T15:57:14.22+0100 [APP/0]      OUT         dialog_request_counter: 7,
2017-02-08T15:57:14.22+0100 [APP/0]      OUT         _node_output_map: [Object] },
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      SQL_Result: '',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      SQL_Request: '',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      Launch_Request: '',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      store: 'Polygone',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      lat: '',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      long: '',
2017-02-08T15:57:14.22+0100 [APP/0]      OUT      Reservation_Number: '' } }
2017-02-08T15:57:14.91+0100 [APP/0]      OUT ----------------------------------------

->  After event 

2017-02-08T15:57:14.91+0100 [APP/0]      OUT message after
2017-02-08T15:57:14.92+0100 [APP/0]      OUT { text: 'How far is polygone',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   user: '1172085072846787',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   channel: '1172085072846787',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   timestamp: 1486565833753,
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   seq: 1567,
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   mid: 'mid.1486565833753:ca0ac6ac76',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT ----------------------------------------
2017-02-08T15:57:14.92+0100 [APP/0]      OUT response after
2017-02-08T15:57:14.92+0100 [APP/0]      OUT { intents: [ { intent: 'store-distance', confidence: 0.5044459165092011 } ],
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   entities: [ { entity: 'store', location: [Object], value: 'Polygone' } ],
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   input: { text: 'How far is polygone' },
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   output:
2017-02-08T15:57:14.92+0100 [APP/0]      OUT    { log_messages: [],
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      text:
2017-02-08T15:57:14.92+0100 [APP/0]      OUT       [ '',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT         'I need to know where your are. Please, click on location icon and send your current position.' ],
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      nodes_visited: [ 'node_1_1483693417283', 'node_2_1483715278541' ] },
2017-02-08T15:57:14.92+0100 [APP/0]      OUT   context:
2017-02-08T15:57:14.92+0100 [APP/0]      OUT    { conversation_id: '2bbecc1a-ce36-4eb1-85ec-d8504ce1547e',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      system:
2017-02-08T15:57:14.92+0100 [APP/0]      OUT       { dialog_stack: [Object],
2017-02-08T15:57:14.92+0100 [APP/0]      OUT         dialog_turn_counter: 8,
2017-02-08T15:57:14.92+0100 [APP/0]      OUT         dialog_request_counter: 8,
2017-02-08T15:57:14.92+0100 [APP/0]      OUT         _node_output_map: [Object] },
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      SQL_Result: '',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      SQL_Request: 'STORE_DISTANCE',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      Launch_Request: '',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      store: 'Polygone',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      lat: '',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      long: '',
2017-02-08T15:57:14.92+0100 [APP/0]      OUT      Reservation_Number: '' } }
2017-02-08T15:57:14.93+0100 [APP/0]      OUT Pas Query Order
2017-02-08T15:57:14.93+0100 [APP/0]      OUT info: Warning: using temporary storage. Data will be lost when process restarts.

-> FB controller Hear 

2017-02-08T15:57:14.93+0100 [APP/0]      OUT controller.hears: message.watsonData
2017-02-08T15:57:14.94+0100 [APP/0]      OUT { intents: [ { intent: 'store-distance', confidence: 0.5044459165092011 } ],
2017-02-08T15:57:14.94+0100 [APP/0]      OUT   entities: [ { entity: 'store', location: [Object], value: 'Polygone' } ],
2017-02-08T15:57:14.94+0100 [APP/0]      OUT   input: { text: 'How far is polygone' },
2017-02-08T15:57:14.94+0100 [APP/0]      OUT   output:
2017-02-08T15:57:14.94+0100 [APP/0]      OUT    { log_messages: [],
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      text:
2017-02-08T15:57:14.94+0100 [APP/0]      OUT       [ '',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT         'I need to know where your are. Please, click on location icon and send your current position.' ],
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      nodes_visited: [ 'node_1_1483693417283', 'node_2_1483715278541' ] },
2017-02-08T15:57:14.94+0100 [APP/0]      OUT   context:
2017-02-08T15:57:14.94+0100 [APP/0]      OUT    { conversation_id: '2bbecc1a-ce36-4eb1-85ec-d8504ce1547e',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      system:
2017-02-08T15:57:14.94+0100 [APP/0]      OUT       { dialog_stack: [Object],
2017-02-08T15:57:14.94+0100 [APP/0]      OUT         dialog_turn_counter: 8,
2017-02-08T15:57:14.94+0100 [APP/0]      OUT         dialog_request_counter: 8,
2017-02-08T15:57:14.94+0100 [APP/0]      OUT         _node_output_map: [Object] },
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      SQL_Result: '',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      SQL_Request: 'STORE_DISTANCE',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      Launch_Request: '',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      store: 'Polygone',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      lat: '',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      long: '',
2017-02-08T15:57:14.94+0100 [APP/0]      OUT      Reservation_Number: '' } }

——————————————————————————————————————————————
Step2: send location
——————————————————————————————————————————————
-> Before event 

2017-02-08T15:58:37.37+0100 [APP/0]      OUT message before
2017-02-08T15:58:37.37+0100 [APP/0]      OUT { text: undefined,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   user: '1172085072846787',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   channel: '1172085072846787',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   timestamp: 1486565916957,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   seq: 1571,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   mid: 'mid.1486565916957:3e3ad77097',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   attachments:
2017-02-08T15:58:37.37+0100 [APP/0]      OUT    [ { title: 'Christophe\'s Location',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT        url: 'https://l.facebook.com/l.php?u=https%3A%2F%2Fwww.bing.com%2Fmaps%2Fdefault.aspx%3Fv%3D2%26pc%3DFACEBK%26mid%3D8100%26where1%3D43.617281187366%252C%2B3.9075159374661%26FORM%3DFBKPL1%26mkt%3Den-US&h=ATPGmVZRTWhD71tA-euv-5jPgH_JhcysD-9SqxVutYhstPsUPgSAeuFueixKwDVvaYBKhpT8l-S9Kzf5tpsJ7JqVajs3A4eTQSt4ZYUJydZD4-YeqixQMOct0PQ_ZImbBKKSsiY&s=1&enc=AZOo44UIBwWeW_Xhwhqj7DuSpefxo5FYixLWFz7zxb5L7IH17WbLsT5Gon93pyGvRUZ5eLL-QgS8rBxYM1DucfEe',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT        type: 'location',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT        payload: [Object] } ],
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   watsonData: { output: { text: [] } } }
2017-02-08T15:58:37.37+0100 [APP/0]      OUT ----------------------------------------
2017-02-08T15:58:37.37+0100 [APP/0]      OUT payload before
2017-02-08T15:58:37.37+0100 [APP/0]      OUT { workspace_id: '934773b3-ed68-43b3-aef2-ad3086ac587a',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   input: { text: undefined },
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   context:
2017-02-08T15:58:37.37+0100 [APP/0]      OUT    { conversation_id: '2bbecc1a-ce36-4eb1-85ec-d8504ce1547e',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      system:
2017-02-08T15:58:37.37+0100 [APP/0]      OUT       { dialog_stack: [Object],
2017-02-08T15:58:37.37+0100 [APP/0]      OUT         dialog_turn_counter: 8,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT         dialog_request_counter: 8,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT         _node_output_map: [Object] },
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      SQL_Result: '',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      SQL_Request: 'STORE_DISTANCE',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      Launch_Request: '',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      store: 'Polygone',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      lat: '',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      long: '',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      Reservation_Number: '' } }
2017-02-08T15:58:37.37+0100 [APP/0]      OUT Location detected. Attachement payload =
2017-02-08T15:58:37.37+0100 [APP/0]      OUT { coordinates: { lat: 43.617281187366, long: 3.9075159374661 } }
2017-02-08T15:58:37.37+0100 [APP/0]      OUT New payload =
2017-02-08T15:58:37.37+0100 [APP/0]      OUT { workspace_id: '934773b3-ed68-43b3-aef2-ad3086ac587a',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   input: { text: 'location' },
2017-02-08T15:58:37.37+0100 [APP/0]      OUT   context:
2017-02-08T15:58:37.37+0100 [APP/0]      OUT    { conversation_id: '2bbecc1a-ce36-4eb1-85ec-d8504ce1547e',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      system:
2017-02-08T15:58:37.37+0100 [APP/0]      OUT       { dialog_stack: [Object],
2017-02-08T15:58:37.37+0100 [APP/0]      OUT         dialog_turn_counter: 8,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT         dialog_request_counter: 8,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT         _node_output_map: [Object] },
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      SQL_Result: '',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      SQL_Request: 'STORE_DISTANCE',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      Launch_Request: '',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      store: 'Polygone',
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      lat: 43.617281187366,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      long: 3.9075159374661,
2017-02-08T15:58:37.37+0100 [APP/0]      OUT      Reservation_Number: '' } }
2017-02-08T15:58:37.96+0100 [APP/0]      OUT ----------------------------------------

-> After event 

2017-02-08T15:58:37.96+0100 [APP/0]      OUT message after
2017-02-08T15:58:37.96+0100 [APP/0]      OUT { text: 'location',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   user: '1172085072846787',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   channel: '1172085072846787',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   timestamp: 1486565916957,
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   seq: 1571,
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   mid: 'mid.1486565916957:3e3ad77097',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   attachments:
2017-02-08T15:58:37.96+0100 [APP/0]      OUT    [ { title: 'Christophe\'s Location',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT        url: 'https://l.facebook.com/l.php?u=https%3A%2F%2Fwww.bing.com%2Fmaps%2Fdefault.aspx%3Fv%3D2%26pc%3DFACEBK%26mid%3D8100%26where1%3D43.617281187366%252C%2B3.9075159374661%26FORM%3DFBKPL1%26mkt%3Den-US&h=ATPGmVZRTWhD71tA-euv-5jPgH_JhcysD-9SqxVutYhstPsUPgSAeuFueixKwDVvaYBKhpT8l-S9Kzf5tpsJ7JqVajs3A4eTQSt4ZYUJydZD4-YeqixQMOct0PQ_ZImbBKKSsiY&s=1&enc=AZOo44UIBwWeW_Xhwhqj7DuSpefxo5FYixLWFz7zxb5L7IH17WbLsT5Gon93pyGvRUZ5eLL-QgS8rBxYM1DucfEe',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT        type: 'location',
2017-02-08T15:58:37.96+0100 [APP/0]      OUT        payload: [Object] } ],
2017-02-08T15:58:37.96+0100 [APP/0]      OUT   watsonData: { output: { text: [] } } }
2017-02-08T15:58:37.96+0100 [APP/0]      OUT ----------------------------------------
2017-02-08T15:58:37.96+0100 [APP/0]      OUT response after
2017-02-08T15:58:37.97+0100 [APP/0]      OUT { intents: [ { intent: 'store-distance', confidence: 1 } ],
2017-02-08T15:58:37.97+0100 [APP/0]      OUT   entities: [],
2017-02-08T15:58:37.97+0100 [APP/0]      OUT   input: { text: 'location' },
2017-02-08T15:58:37.97+0100 [APP/0]      OUT   output:
2017-02-08T15:58:37.97+0100 [APP/0]      OUT    { log_messages: [],
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      text: [ 'I understand that you are here: latitude 43.617281187366 & longitude 3.9075159374661' ],
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      nodes_visited: [ 'node_6_1483719981993' ] },
2017-02-08T15:58:37.97+0100 [APP/0]      OUT   context:
2017-02-08T15:58:37.97+0100 [APP/0]      OUT    { conversation_id: '2bbecc1a-ce36-4eb1-85ec-d8504ce1547e',
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      system:
2017-02-08T15:58:37.97+0100 [APP/0]      OUT       { dialog_stack: [Object],
2017-02-08T15:58:37.97+0100 [APP/0]      OUT         dialog_turn_counter: 9,
2017-02-08T15:58:37.97+0100 [APP/0]      OUT         dialog_request_counter: 9,
2017-02-08T15:58:37.97+0100 [APP/0]      OUT         _node_output_map: [Object] },
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      SQL_Result: '',
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      SQL_Request: 'STORE_DISTANCE',
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      Launch_Request: '',
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      store: 'Polygone',
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      lat: 43.617281187366,
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      long: 3.9075159374661,
2017-02-08T15:58:37.97+0100 [APP/0]      OUT      Reservation_Number: '' } }
2017-02-08T15:58:37.97+0100 [APP/0]      OUT Pas Query Order
2017-02-08T15:58:37.97+0100 [APP/0]      OUT info: Warning: using temporary storage. Data will be lost when process restarts.

-> No FB controller Hear

Step 1: OK

message received
Conversation understands
message sent to FB messenger (FB controller Hear)

Step 2: KO

message received and updated (conversationPayload.input.text = "location";)
Conversation understands
no message sent to FB messenger (no FB controller Hear)

stevenoh93 commented 7 years ago

Sorry about the late response again. Let me try this myself and get back to you

stevenoh93 commented 7 years ago

It looks like others are experiencing the same problem:
The problem is that message_received event is not being fired when a user sends the location. So the middleware is responding to the incoming message, but the .hears function is not being fired, therefore, no response is being delivered. Seems like someone posted a workaround in the issue I mentioned above.

emrhio commented 7 years ago

Has this issue been resolve? I'm trying the same and when i do: controller.hears('(.*)','message_received', function(bot, message) {... and send any attachment before nor hears gets fire,

And when i do: controller.on(message_received', function(bot, message) {...

on gets fire, but before does not. It never enters to middleware.before...

Any ideas?

germanattanasio commented 7 years ago

@stevenoh93 any update on this?

stevenoh93 commented 7 years ago

@germanattanasio I'm not with the team anymore and it's been a while, so I don't quite have the context to answer this question fully. Let me give it a shot anyway.

@clalevee Judging from this thread, it seems like you have to update the list of events to listen for. When you send an attachment, it doesn't look like message_received event is fired. Not sure which event is fired when receiving an attachment, but you have to listen for that event. You can check the FB documentation.

germanattanasio commented 6 years ago

@Naktibalda what do you think about this? I don't think it's a problem with this library but something specific of botkit

Naktibalda commented 6 years ago

Yes, it looks specific to botkit's integration with Facebook.

Naktibalda commented 6 years ago

Here is an open botkit issue with suggested workaround: https://github.com/howdyai/botkit/issues/1054

germanattanasio commented 6 years ago

I'm closing this then. Feel free to re-open if you think it's related to this library.

HARVS1789UK commented 5 years ago

@germanattanasio and @Naktibalda I am experiencing either an identical or very similar issue to this, I have read through all of the comments here and followed and read the linked fixes/related issues in Botkit repos etc, none of which seem to resolve the issue for me.

Should I raise a new issue, or could this issue be reopened?

To summerise:

.hears() never gets triggered (understandable as Facebook sends a message with no text and a single attachment when delivering user location data)
.on('message_received') does get triggered, but even if updating the message/event object to set text: 'LOCATION_DATA_RECEIVED no further middleware code is executed (i.e. Watson never gets sent the message)
I therefore call sendToWatsonAsync(), passing my modifed message and contextDelta, in .on(). This completes successfully and there are numerous communications between my codebase and Watson (including a chain of dialog actions).
I can see all of the correct responses and context being set in my console, but none of these messages returned to the middleware from Watson are ever sent onward to Facebook (basically the entire conversation continues as expected in my console logs....but none of it is ever sent back to the end user)

Things which I think might be related are:

The following lines of index.js mean an empty-ish watsonData property is added to my FB location message -

  if ((!message.text && message.type !== 'welcome') || ignoreType.indexOf(message.type) !== -1 || message.reply_to || message.bot_id) {
    // Ignore messages initiated by Slack. Reply with dummy output object
    message.watsonData = {
      output: {
        text: []
      }
    };
    return next();
  }

middleware.sendToWatsonAsync() has no return value, so once I have called it in .on() I cannot manually call bot.reply() to pass back the Watson responses to FB

Very happy to discuss with someone in real time on Slack or Skype etc to explain further and/or provide access to my codebase is that would help.

Naktibalda commented 5 years ago

This looks like a long way to say, that you hadn't found a way to call bot.reply.

Here is example code from README:

      checkBalanceAsync(message.watsonData.context).then(function (contextDelta) {
        return watsonMiddleware.sendToWatsonAsync(bot, newMessage, contextDelta);
      }).catch(function (error) {
        newMessage.watsonError = error;
      }).then(function () {
          // call bot.reply here
      });

HARVS1789UK commented 5 years ago

@Naktibalda - The issue is I don't have access to the watson response object (or message) to use in any bot.reply().

I could very easily add bot.reply(newMessage, 'Hello World'); or something equally as static, but what I need to respond with is numerous replies which have all been formed by Watson in response to me manually updating the incoming message object and sending off an updated context to Watson.

To attempt to illustrate what I am talking about, the conversation flow goes something like:

Bot: "I'm a chatbot who can help you plan a bus journey, where are you trying to get to?" User: [shared location]

I now pull out the lat/lng from the FB message location object, use Google Maps API to get a place/street name and pass both across to Watson in context.latest_user_location

Bot: "OK, I will find some bus stops near there..." Bot: "I have found the following stops:" Bot: [Facebook Image attachment] Bot: [Facebook List Template listing 4 x bus stops with 'select' buttons for each] User: [selects a stop]

The dynamic look ups of nearby bus stops and static Google map images etc are all managed via 'Dialog Actions' and basically require Watson and the codebase to bounce between each other a couple of times before finally responding to the end user.

It is the end result(s) of this that I need to send back to the user (rather than a hardcoded response via bot.reply() and unless I am mistaken, sendToWatsonAsync does not return anything, so I cannot just wait for it complete and then call bot.reply(res.message, res.message.text.join('\n')?

That wouldn't be an issue if every time the response from Watson to watsonMiddleware (in app.js) forwarded things on to the Facebook controller (like regular Watson responses do), but in this scenario (either when the initial user input is location sharing from FB, or maybe just whenever calling sendToWatsonAsync manually, I am not sure which it is yet) that never happens, so my middleware.after = function(message, watsonRequest, callback) print out the entire conversation as expected to the console.log but nothing is sent back to Facebook.

Am I missing something obvious?

Naktibalda commented 5 years ago

syncToWatsonAsync returns promise, If you prefer callback style, use syncToWatson.

Naktibalda commented 5 years ago

It is sendToWatson by the way.

HARVS1789UK commented 5 years ago

@Naktibalda - My mistake, syncToWatson was just my typo/writing the method name from memory. I have corrected it in my above comment.

Regarding the lack of return value from sendToWatsonAsync or sendToWatson, I understand that the async version returns a promise, I am using it in a promise/then-able chain as per the below:

            // Add location name to contextDelta and send location data to Watson
            Promise.resolve(
                getLocationByCoords({ location: { coords: latlng }})
            )
                .then(function(results) {
                    contextDelta.latest_location_data.name = results.context;
                    return watsonMiddleware.sendToWatsonAsync(bot, newMessage, contextDelta);
                })
                .then(function(response) {
                    console.log("Completed syncToWatsonAsync()");
                    console.log(response);
                    //bot.reply(newMessage, "Something Dynamic...");
                    return;
                })
                .catch(function(error) {
                    console.log(error);
                    bot.reply(newMessage, 'I\'m sorry, but for technical reasons I can\'t respond to your message')
                });

However, console.log(response) just prints undefined which is what I mean when I say that sendToWatson doesn't have a return value. The async version returns a promise yes, but when that promise resolves it just returns void/nothing?

This is why my bot.reply() line below is commented out or can currently only be something static.

I have tried the regular synchronous approach as well, I can't remember what exactly the results of that were, but they weren't successful either.

Given your suggestion and my code snippet above, how can I access the updated conversational data from Watson in that context?

Naktibalda commented 5 years ago

Watson response is added to message object, so it is in newMessage.watsonData.

HARVS1789UK commented 5 years ago

@Naktibalda Thank you so much! I had not considered/understood that the newMessage object I pass to sendToWatsonAsync would be updated 'in place' or 'by reference' however you want to term it (I am not that well versed in JS/NodeJS), so I was basically trying to do the same thing but using a return value instead of the original message object.

I now have this and everything is functioning as desired:

                .then(function(response) {
                    console.log("Completed syncToWatsonAsync()");
                    sendResponse(newMessage);
                    return;
                })

(where send response is a method of mine which either does bot.reply()or bot.startConversation()depending on how many entries their are in message.watsonData.output.generic).

I guess the only outstanding question/issue then (which might well be why this issue was raised initially) is why does my:

controller.hears('(.*)', ['facebook_postback', 'message_received'], function(bot, message) { }

listener not get triggered during/after my manual call to sendToWatsonAsync and deal with sending the responses back to Facebook automatically (as it does in all other contexts when responses come back from Watson)?

Effectively I have only needed to add a manual call to sendResponse(newMessage); because the .hears() listener hasn't been triggered oin this context.

watson-developer-cloud / botkit-middleware

Handle user location #27