microsoft / botframework-sdk

Bot Framework provides the most comprehensive experience for building conversation applications.
MIT License
7.5k stars 2.44k forks source link

[Skype] Error 502: Bad Gateway when sending proactive message to user via saved message.address #3312

Closed xitij closed 7 years ago

xitij commented 7 years ago

System Information

Bot Info

Issue Description

At least once a day we push out a news item to our bot users. We store conversation ids in our db and attempt to send a message to the users. We have one service which uses Kue to queue a message and another service that peels the messages from the queue and attempts to send them to users. After calling session.send() we receive the following error (not always but this happen quite often):

Aug 11 11:09:10 gameon-bot-prd app/dispatch.1: Error: Request to 'https://smba.trafficmanager.net/apis/v3/conversations/29%3A1JlbBv71PBqo_oaJAk7iKPSUbEDMiqWzUjqhiWbmYflax5TIbnKyHAmrli2gTawOQ/activities' failed: [502] Bad Gateway 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at Request._callback (/app/node_modules/botbuilder/lib/bots/ChatConnector.js:523:46) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at Request.self.callback (/app/node_modules/request/request.js:188:22) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at emitTwo (events.js:106:13) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at Request.emit (events.js:191:7) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at Request.<anonymous> (/app/node_modules/request/request.js:1171:10) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at emitOne (events.js:96:13) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at Request.emit (events.js:188:7) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1091:12) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at IncomingMessage.g (events.js:291:16) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at emitNone (events.js:91:20) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at IncomingMessage.emit (events.js:185:7) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at endReadableNT (_stream_readable.js:974:12) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at _combinedTickCallback (internal/process/next_tick.js:74:11) 
Aug 11 11:09:10 gameon-bot-prd app/dispatch.1:     at process._tickCallback (internal/process/next_tick.js:98:9) 

Code Example

This is code from the service that peels out of the queue and attempts to send to the user.

const dispatch = (job, done) => {
  const { appId, address } = job.data;
  const messages = job.data.messages.map(message => message.address(address));

  async.eachSeries(messages, (message, callback) => {
    const channelId = message.address.channelId;
    const channel = (channelId === 'facebook' || channelId === 'skype') ? channelId : 'other';
    bots[appId][channel].send(message, callback);
  }, done);
};

Promise.map(['facebook', 'skype', 'kik', 'telegram', 'slack'], (platform) => (queue.process(platform, concurrency, dispatch)));

Steps to Reproduce the Issue

  1. session.send() with an IMessage
  2. ...??
  3. Receive [502]: "Gateway Error" from "smba.trafficmanager" which I believe is Skype.

Expected Behavior

Not get an error, or at least better determine what the error is so that I can correct it.

Actual Results

See above error

nwhitmont commented 7 years ago

Hi @xitij - I noticed you are using an older version of Bot Builder SDK.

Are you still experiencing the issue after upgrading to the current 3.9.0 version? https://www.npmjs.com/package/botbuilder

nwhitmont commented 7 years ago

Instead of saving the conversationId, you need to save the session.message.address for each user that has interacted with your bot. If you try to send to a save conversationId, it won't work since the conversation ID is generated by each channel and can change or expire when your user starts a new conversation with the bot.

For more information on proactive messages, see: https://docs.microsoft.com/en-us/bot-framework/nodejs/bot-builder-nodejs-proactive-messages#send-an-ad-hoc-proactive-message

You can find a Node.js code example of sending proactive messages here: https://github.com/Microsoft/BotBuilder-Samples/tree/master/Node/core-proactiveMessages

xitij commented 7 years ago

Instead of saving the conversationId, you need to save the session.message.address for each user that has interacted with your bot. If you try to send to a save conversationId, it won't work since the conversation ID is generated by each channel and can change or expire when your user starts a new conversation with the bot.

Sorry we are saving the full address in our db. I misspoke. I'll try upgrading to Bot Builder v3.9.0 and report back.

xitij commented 7 years ago

@nwhitmont I upgraded to v3.9.0 and I'm still seeing the issue.

xitij commented 7 years ago

Bump. Still waiting for some feedback here.

nwhitmont commented 7 years ago

@xitij Before you send the proactive message, did you construct a message object using the saved address?

Example:

var bot = new builder.UniversalBot(connector);

function sendProactiveMessage(address) {
    var msg = new builder.Message().address(address);
    msg.text('Hello, this is a notification');
    msg.textLocale('en-US');
    bot.send(msg);
}

Otherwise, it's hard to say what the problem is without seeing your complete bot code, specifically how you are configuring the message object before sending it.

Can you post your complete bot code here or in a shared repository?

xitij commented 7 years ago

@nwhitmont Yes I create a message object. Here's closer to the important parts of full code. I'm not able to post the full code base. I'd like to add that this works on multiple platforms: Facebook, Skype, Kik, Telegram, and Slack. These platforms all send (mostly) successful proactive messages (including Skype) so this doesn't fail all the time. The main problem/question here is why I get this error at times from Skype, and what this error actually means. I've not really been able to find an explanation because Skype doesn't have documentation and you're required to use Bot Framework for Skype bots which obfuscates the entire API.

This is dispatch.js which does all the heavy lifting of sending the messages.

const channels = ['facebook', 'skype', 'other'];

mongo.connect(mongoUrl, { promiseLibrary: Promise })
  .then(db => (
    db.collection('apps').find().toArray()
      .map(app => {
        const connector = new builder.ChatConnector({
        appId: app.id,
        appPassword: sjcl.decrypt(secret, app.password)
      });
      // this returns an array of arrays
      return channels.map(channel => ({
        app,
        bot: new builder.UniversalBot(connector, { persistConversationData: true }),
        channel
      }));
    })
    .reduce((result, data) => {
      // which is flatten to an array here  
      data.forEach((item) => (result.push(item)));
      return result;
    }, [])
    .reduce((result, data) => {
      // which is turned into an object here
      if (!result[data.app.id]) {
        result[data.app.id] = {};
      }
      result[data.app.id][data.channel] = data.bot;
      return result;
    }, {}).then(bots => {
      const dispatch = (job, done) => {
        const { appId, address } = job.data;
        const messages = job.data.messages.map(message => (new Formatter().create({ message, address }).format());

        async.eachSeries(messages, (message, callback) => {
          const channelId = message.address.channelId;
          const channel = (channelId === 'facebook' || channelId === 'skype') ? channelId : 'other';
          bots[appId][channel].send(message, callback);
        }, done);
      };

    Promise.map(['facebook', 'skype', 'kik', 'telegram', 'slack'], (platform) => (queue.process(platform, concurrency, dispatch)));
  })
}).catch((err) => {
  console.log(err);
});

This dispatch service is running at all times waiting for items in the queue to dispatch. queue is a redis queue using this library.

Here's the code for Formatter:

const Formatter = (function() {
  this.create = (params) => {
    let formatter;
    let { address } = params;
    const { session, message } = params;
    address = address || session.message.address;
    formatter = new BaseFormatter(session, message, address);
    return formatter;
  };
};

const BaseFormatter = (function() {
  function BaseFormatter(session, msg, address) {
    this.address = address;
    this.session = session;
    this.input = {};
    this.input.msg = msg;
  }

  BaseFormatter.prototype.createBotMessage = function() {
    if (this.session) {
      return new builder.Message(this.session).address(this.address);
    }
    return new builder.Message().address(this.address);
  };

  BaseFormatter.prototype.format_text = function() {
    let msg = this.createBotMessage()
      .text(this.input.msg.text.replace(new RegExp('\n', 'g'), '<br/>'));
    return msg;
  };

Finally messages are constructed in relay.js pushed into the above queue where they are dequeued and sent by dispatch.js:

function handleMessage(conf) {
  const { db, app, msg, topic, handler } = conf;

  // All users are sent the same message
  return db.getAddresses()
    .toArray()
    .map((address, index) => ({ message: msg, address }))
    .mapSeries((item) => {
      const { address, message } = item;
      const relay = { address, messages: [message], appId: app.id };
      return new Promise((resolve, reject) => {
        const job = queue.create(address.channelId, relay)
          .removeOnComplete(true)
          .save(err => (err ? reject(err) : resolve()))
          .on('failed', (errorMessage) => {
            console.error(`${app.id} : ${address.id} : ${errorMessage} : ${job.id}`);
            job.remove(err => {
              if (err) {
                console.error(`error removing job:${job.id}`);
              }
            });
          })
      });
    });
}

function subscribe(db, app) {
  const channel = `/huddles/${app.huddleId}/messages`;
  const eventHandler = EventHandler({ db });
  console.log(`Subscribing to channel: ${channel}`);
  // this subscribes to some endpoint that will receive messages we want to send to all users
  subscribe({
    channel,
    message(message) {
      handleMessage({ db, app, msg: message.data, handler: eventHandler });
    },
    connect(chn) {
      pino.info(`Subscribed to PubNub channel: ${chn}`);
    }
  });
  return null;
}

MongoClient.connect(mongoUrl)
  .then((mongo) => mongo.collection('apps').find().toArray())
  .mapSeries((app) => mongoDB.connect(app.id).then((db) => subscribe(db, app)))
  .catch((e) => {
    pino.error('Failed to set up mongo: ', e);
    process.exit(1);
  });
nwhitmont commented 7 years ago

Like you said in your last comment... "this doesn't fail all the time". So it's possible you are hitting random service interruptions. Keep in mind the Bot Framework is in Preview state and under active development so sometimes things might not work.

Open a new issue if you are still experiencing problems.