search

jgorset / facebook-messenger

Definitely the best way to make Bots on Facebook Messenger with Ruby
MIT License
962 stars 211 forks source link

Seeing duplicate messages from sender #46

Closed timfong888 closed 8 years ago

timfong888 commented 8 years ago

I instrumented the webhook URL. I see two entries for every one input from a user.

The payload appears to be exactly the same in terms of time stamp and so forth. So I am assuming it's not the user or the client submitting something twice.

It appears to be somewhat intermittent as well. Sometimes only one input will be processed. But mostly it is a duplicate or triplicate.

jgorset commented 8 years ago

That's really weird. Does unsubscribing and subscribing to the Facebook Page again help?

timfong888 commented 8 years ago

I can check.

I instrumented it -- the client is definitely sending only one request. But the Messenger server is sending 2 or more to the webhook URL -- with exactly the same payload.

Here is a sample payload -- I will revoke it after you see it:

https://www.runscope.com/public/3d4879fd-acd1-438f-a484-3b88af1105a5/fc3ad64e-0752-434d-a706-f9dfe5920ccc

timfong888 commented 8 years ago

I also am getting a notification in the Facebook developers dashboard:

Webhooks Failing

timfong888 commented 8 years ago

unsubscribing and resubscribing didn't fix it.

frodsan commented 8 years ago

You must send back a 200, within 20 seconds, to let us know you've successfully received the callback. Otherwise, the request will time out.

When they don't receive a 200, they will send one or multiple requests to the webhook again. Can you check if this could be your problem?

UPDATE: I think that's why you are getting those notifications from Facebook. I get those when my webhook returns other status code than 200 for a while.

timfong888 commented 8 years ago

While there does sometimes to be times there is no 200 response, in this case, each response was a 200, yet the same one got sent multiple times.

Your Messenger server sends the 200 right upon receipt of the request, that's not something my app does, right?

timfong888 commented 8 years ago

BTW, I cannot seem to proxy calls your Bot makes to the graph.facebook.com end-point even when I change Bot.base_uri to a proxy. I am guessing that it gets reset with the "deliver" method?

jgorset commented 8 years ago

@frodsan and I discussed that it could be that your bot is taking over 20 seconds to process a message, which would cause Facebook to consider it a failed request according to the documentation:

If your webhook returns an error (i.e., not a 2XX status) or times out (i.e., takes longer than 20 second to respond) for over 15 minutes, then we will send you a warning developer alert.

Do you think that could be the case? That would explain both the warnings you're getting on your dashboard and why you're getting messages twice.

timfong888 commented 8 years ago

Is there a way to respond with a 200 and then use the deliver method to send a text response?

On Thursday, June 23, 2016, Johannes Gorset notifications@github.com wrote:

@frodsan https://github.com/frodsan and I discussed that it could be that your bot is taking over 20 seconds to process the message, which would cause Facebook to consider it a failed request according to the documentation https://developers.facebook.com/docs/messenger-platform/webhook-reference#unsubscribe :

If your webhook returns an error (i.e., not a 2XX status) or times out (i.e., takes longer than 20 second to respond) for over 15 minutes, then we will send you a warning developer alert.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hyperoslo/facebook-messenger/issues/46#issuecomment-227998160, or mute the thread https://github.com/notifications/unsubscribe/AAA256O1CMtlDCThAorsHqlMZCDgN4V7ks5qOlMLgaJpZM4I6P-W .

Sent from Gmail Mobile

jgorset commented 8 years ago

Sure! If processing a message and responding to it takes more than 20 seconds, you could (and indeed probably should) process it in the background. You can use a background processing library like Sidekiq to make that easier.

Another option is changing this gem to be asynchronous and just responding with HTTP 200 immediately, but that adds unnecessary complexity for most bots (asynchronous programming is hard in Ruby and you'd miss out on Facebook retrying failed messages for bots that are quick enough to respond within the 20 seconds). I don't think we should do that. What do you think, @frodsan and @nashby?

timfong888 commented 8 years ago

Thanks. How would processing in the background allow a response in under a few seconds. The #deliver is still in the Bot.listen loop. I assume messenger waits for #deliver to know there is a response?

On Sunday, June 26, 2016, Johannes Gorset notifications@github.com wrote:

Sure! If processing a message and responding to it takes more than a few seconds, you could (and indeed probably should) process it in the background. You can use a background processing library like Sidekiq http://sidekiq.org/ to make that easier.

Another option is changing this gem to be asynchronous and just responding with HTTP 200 immediately, but that adds unnecessary complexity for most bots (asynchronous programming is hard in Ruby and you'd miss out on Facebook retrying failed requests for bots that are quick enough to respond within the 20 seconds).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hyperoslo/facebook-messenger/issues/46#issuecomment-228592788, or mute the thread https://github.com/notifications/unsubscribe/AAA250Qk8HNVMdGR5CKF5yMdMvG9u0Mtks5qPkmugaJpZM4I6P-W .

Sent from Gmail Mobile

timfong888 commented 8 years ago

On reflection I think I see. You mean within the #listen block to kick off the asynchronous job of processing the response and then delivering the message.

When do you send the 200? I can remove the #deliver and it just responds with 200 when it completes the #listen block?

On Sunday, June 26, 2016, Johannes Gorset notifications@github.com wrote:

Sure! If processing a message and responding to it takes more than 20 seconds, you could (and indeed probably should) process it in the background. You can use a background processing library like Sidekiq to make that easier.

Another option is changing this gem to be asynchronous and just responding with HTTP 200 immediately, but that adds unnecessary complexity for most bots (asynchronous programming is hard in Ruby and you'd miss out on Facebook retrying failed requests for bots that are quick enough to respond within the 20 seconds). I don't think we should do that. What do you think, @frodsan https://github.com/frodsan and @nashby https://github.com/nashby?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hyperoslo/facebook-messenger/issues/46#issuecomment-228592817, or mute the thread https://github.com/notifications/unsubscribe/AAA25zStUIAsdL2-tmAElaxwIVDIaduWks5qPknZgaJpZM4I6P-W .

Sent from Gmail Mobile

timfong888 commented 8 years ago

I got it to work. I send an async job now so the 200 comes in under 200ms.

So far so good.

As to changing the end point to a proxy: is the only option now to modify the gem locally?

It seems I can access it but it reverts to the original uri when I use the

deliver method.

On Sunday, June 26, 2016, Timothy Fong timfong888@gmail.com wrote:

On reflection I think I see. You mean within the #listen block to kick off the asynchronous job of processing the response and then delivering the message.

When do you send the 200? I can remove the #deliver and it just responds with 200 when it completes the #listen block?

On Sunday, June 26, 2016, Johannes Gorset <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Sure! If processing a message and responding to it takes more than 20 seconds, you could (and indeed probably should) process it in the background. You can use a background processing library like Sidekiq to make that easier.

Another option is changing this gem to be asynchronous and just responding with HTTP 200 immediately, but that adds unnecessary complexity for most bots (asynchronous programming is hard in Ruby and you'd miss out on Facebook retrying failed requests for bots that are quick enough to respond within the 20 seconds). I don't think we should do that. What do you think, @frodsan https://github.com/frodsan and @nashby https://github.com/nashby?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hyperoslo/facebook-messenger/issues/46#issuecomment-228592817, or mute the thread https://github.com/notifications/unsubscribe/AAA25zStUIAsdL2-tmAElaxwIVDIaduWks5qPknZgaJpZM4I6P-W .

Sent from Gmail Mobile

Sent from Gmail Mobile

jgorset commented 8 years ago

You should be able to open the Facebook::Messenger::Bot class and change its base_uri in your application:

class Facebook::Messenger::Bot
  base_uri 'https://your.proxy'
end

Why do you need that, though?

timfong888 commented 8 years ago

So I can instrument it for troubleshooting to see how the API is responding and what is the pylons being sent.

On Monday, June 27, 2016, Johannes Gorset notifications@github.com wrote:

You should be able to open the Facebook::Messenger::Bot https://github.com/hyperoslo/facebook-messenger/blob/master/lib/facebook/messenger/bot.rb class and change its base_uri in your application:

class Facebook::Messenger::Bot base_uri 'https://your.proxy'end

Why do you need that, though?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hyperoslo/facebook-messenger/issues/46#issuecomment-228668352, or mute the thread https://github.com/notifications/unsubscribe/AAA25-B_9LfVzhJ2WDLFW8cU86ZvP7Z6ks5qP3WhgaJpZM4I6P-W .

Sent from Gmail Mobile

jgorset commented 8 years ago

That's a pretty good reason. In fact, if you want to make a configuration option for that make sure we don't break that in the future then I'd be happy to accept that pull request!

jgorset commented 8 years ago

Either way, I'm closing this issue as resolved.

Vuta commented 6 years ago

hi @timfong888 , may I know how did you make a async job to respond 200 to fb under 20s please? I'm having this trouble too and can't figure out a way to fix it

nicolrx commented 6 years ago

Hi @Vuta, I think he does the following: -Instead of performing tasks related to the parsed message within the payload block, he performs those tasks asynchronously in a worker (eg. with Sidekiq) -Then, he just delivers the response while the workers are doing their job => Thus, the response time is way faster

timfong888 commented 6 years ago

Yep!

On Sat, Feb 10, 2018 at 2:33 PM nico_lrx notifications@github.com wrote:

Hi @Vuta https://github.com/vuta, I think he does the following: -Instead of performing tasks related to the parsed message within the payload block, he performs those tasks asynchronously in a worker (eg. with Sidekiq) -Then, he just delivers the response while the workers are doing their job => Thus, the response time is way faster

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jgorset/facebook-messenger/issues/46#issuecomment-364659088, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA25wNLBoPacYbh7urjZuCApxKWvQS3ks5tTaicgaJpZM4I6P-W .

-- Sent from Gmail Mobile