postRedditModmailReply ConnectionError causes duplicate modmail entry

[erocs]

This connection error caused a duplicate message to be sent to modmail. I'm mostly just documenting this. I don't think that it's a big deal. Aside from checking the last modmail reply to see if the message was already sent, there isn't much which could be done.

This has happened 2 times now.

Processing Request Tracker Replies to ModMail. Found CommentId for ticketId Reddit main modmail reply url is 'http://www.reddit.com/message/messages/xxxxx' Sending modmail reply to redditurl http://www.reddit.com/message/messages/xxxxx: Thanks for the report. It's handled. 2015-05-06 07:08:09.100000 - Error when attempting to process modmail replies on line number 657. Exception: <class 'requests.exceptions.ConnectionError'> * print_tb: File "modmail_ticketmanager.py", line 657, in processRequestTrackerRepliesToModMail processTicketModmailReply(ticketId, reply, r) * print_exception: Traceback (most recent call last): File "modmail_ticketmanager.py", line 657, in processRequestTrackerRepliesToModMail processTicketModmailReply(ticketId, reply, r) File "modmail_ticketmanager.py", line 701, in processTicketModmailReply postRedditModmailReply(redditUrl, replyText, prawContext) ConnectionError: ('Connection aborted.', ResponseNotReady()) * print_exc: Traceback (most recent call last): File "modmail_ticketmanager.py", line 657, in processRequestTrackerRepliesToModMail processTicketModmailReply(ticketId, reply, r) File "modmail_ticketmanager.py", line 701, in processTicketModmailReply postRedditModmailReply(redditUrl, replyText, prawContext) File "modmail_ticketmanager.py", line 713, in postRedditModmailReply message.reply(full_reply_text) File "/usr/local/lib/python2.7/dist-packages/praw/objects.py", line 384, in reply response = self.reddit_session._add_comment(self.fullname, text) File "/usr/local/lib/python2.7/dist-packages/praw/decorators.py", line 338, in wrapped return function(cls, _args, _kwargs) File "/usr/local/lib/python2.7/dist-packages/praw/init.py", line 2173, in _add_comment retry_on_error=False) File "/usr/local/lib/python2.7/dist-packages/praw/decorators.py", line 163, in wrapped return_value = function(reddit_session, _args, _kwargs) File "/usr/local/lib/python2.7/dist-packages/praw/init.py", line 561, in request_json retry_on_error=retry_on_error) File "/usr/local/lib/python2.7/dist-packages/praw/init*.py", line 402, in _request response = handle_redirect() File "/usr/local/lib/python2.7/dist-packages/praw/init.py", line 375, in handle_redirect timeout=timeout, _kwargs) File "/usr/local/lib/python2.7/dist-packages/praw/handlers.py", line 144, in wrapped result = function(cls, _kwargs) File "/usr/local/lib/python2.7/dist-packages/praw/handlers.py", line 54, in wrapped return function(cls, _kwargs) File "/usr/local/lib/python2.7/dist-packages/praw/handlers.py", line 99, in request allow_redirects=False) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send r = adapter.send(request, *_kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 415, in send raise ConnectionError(err, request=request) ConnectionError: ('Connection aborted.', ResponseNotReady())

[Pentom]

Happened twice? Well thats unfortunate.

Let me consider a bit. Wonder if there is a way to handle this in the search request itself.

How often is this occuring? IE - X Percent of the time?

[Pentom]

No easy fix but there is one if you recognize that modmail replies occur rarely compared to other actions.

• On finding a modmail reply:
• Get the ticket history via response = resource.get(path='ticket/' + str(ticketId) + '/history?format=l')
• Find the last history id where the modmail reply field was changed to this.
• Find if there exists any pulled-in comments from reddit that match this modmail reply.
• Conditionally add new reddit reply based on existence or non-existence of aforementioned reply.

I will probably add this but looking for how often this is occuring? IE - Out of 100 modmail replies, this happened so far ____ times. Unless its so incredibly unbelievably rare, its got to be fixed.

[erocs]

It'll be hard to tell exactly how often as it's only when I notice it that I can report it. I did just see an instance of the other direction duplicating; a post that a player made in modmail was duplicated into RT.

[Pentom]

Ok. Will look into adding this extra safety on ticket -> modmail. I don't know why you had a duplicate message from modmail->Ticket system though. Mind looking into the actual modmail queue and see if that wasnt someone just putting the message twice?

[erocs]

Lol, you're right. They did make a double post. So right now it's just the ticket -> modmail issue.

On Wed, May 6, 2015 at 1:59 PM, Pentom notifications@github.com wrote:

Ok. Will look into adding this extra safety on ticket -> modmail. I don't know why you had a duplicate message from modmail->Ticket system though. Mind looking into the actual modmail queue and see if that wasnt someone just putting the message twice?

— Reply to this email directly or view it on GitHub https://github.com/Pentom/modmail_ticketmanager/issues/8#issuecomment-99608447 .

[Pentom]

Yeah, I got this then. Should be completed sometime before eod.

You can accept the earlier fix in isolation if you want to get that unicode message imported sooner rather than later, or can wait for this one also.

[erocs]

I can wait. All the other fix is holding up is importing additional historical posts.

On Wed, May 6, 2015 at 2:15 PM, Pentom notifications@github.com wrote:

Yeah, I got this then. Should be completed sometime before eod.

You can accept the earlier fix in isolation if you want to get that unicode message imported sooner rather than later, or can wait for this one also.

— Reply to this email directly or view it on GitHub https://github.com/Pentom/modmail_ticketmanager/issues/8#issuecomment-99612216 .

[Pentom]

Setting changes:

• Updated setting redditSleepIntervalInSecondsBetweenRequests from 5 seconds to 60 seconds. Strongly suggest you do the same.
• Look at the new tokenized replies and consider if you wish to change these. Note subject is not valid for more than the initial ticket/subject and not for general replies. Only token for the previous requestTrackerRedditModmailReply reamins as it is - {Content} - and this has not changed.
• Add the three new settings requestTrackerInitialTicketCreationSubject, requestTrackerInitialTicketCreationComment, requestTrackerThreadReply

Reason for sleep interval is easy - an extra 55 seconds between executions is not going to hurt anyone and I have personally seen it take ~30 seconds for reddit to 'insert' a reply that we absolutely inserted. Just means we may not get the reply back immediately if we go from making a modmail reply to checking for new reddit replies - would make the new feature (additional checking here) useless. The 60 seconds should be enough time to add another safety net here. Change to check for existence seems to work with my test cases - should lower incident of duplication but absolutely will not 100% prevent it - I am shooting for something we notice every few weeks instead of a few times in a week.

[erocs]

I was already running the script with a 30 second interval. The lag time to see that the bot has made a reply will be more noticeable, but people will adjust.