Consider detecting when user claims the bot's to be working incorrectly

[Martoon-00]

Clarification and motivation

Correctly parsing the time references from the user's messages is a very difficult task that may require many polishing iterations.

Thus I suggest leaving an easy way for a user to say: "Time bot didn't react to my message" or "Time bot converted the time incorrectly".

If this initiative makes sense, we should do the following:

• Add IssueReportReference type
• Parse all user messages into such issue reports too. Format should be flexible, some "Time zone bot don't work" or "Tzbot is working incorrectly" should be recognized.
• When finding such a message, we should get a few prior messages to know on what message the timebot was expected to react differently.
  • I propose picking 5 prior messages.
  • If the complaining message is in a thread, let's pick 5 prior messages within that thread.
  • If the complaining message is in a thread and there are less than 5 messages before it, then let's obtain those prior messages + the root message that started the thread.
  • In either case, let's also pick the id of the complaining message.
• Those messages should be logged/saved by the bot.
  • For now I propose writing a message about the incident to a dedicated #tzbot-feed channel. Make sure you invite @Martoon-00 , @dcastro and other team members who will be working at tzbot at the time.

Acceptance criteria

• We collect users' complains about time bot working incorrectly.
• We have a nice way to report those incidents to the team and interested team-leads.

[YuriRomanowski]

I think that we should try to record only really wrongly processed messages, because it will be rather difficult to filter them out if they are mixed with correctly processed ones. What can be reasonable to consider:

• Support a slash-command (for example `/report) that a user can use to give a feedback; we can support some forms of it:
  • /report https://serokell.slack.com/archives/C04EKQYLBRC/p1672315874242889 doesn't handle time shifts;
  • /report "tomorrow at 10:00" tomorrow is not 12th December;
  • /report -1 <...> (one message back, but I'm not sure how to fetch it from Slack); Commentary is optional, taking place from the link/message itself to the line end.
• Support /help command, or /report help, showing a help message to the person who wants to help us.
• From time to time, randomly, add some information to the time-translating messages, or just post separate messages to channels several times a day.

As for how to record the feedback left, do you propose to write directly to the serokell owned channel? I think it's reasonable to use while our bot is just our internal tool, later we will have to eliminate this feature I think. But we can just log these messages to a simple file, separately from other logs, and later we can process these files.

Does it make sense?

[Martoon-00]

I agree that we better report only bad messages.

/report functionality with ability to specify the exact reason sounds awesome, much better than what I proposed with automatic detection.

From time to time, randomly, add some information to the time-translating messages, or just post separate messages to channels several times a day.

Which information?

As for how to record the feedback left, do you propose to write directly to the serokell owned channel?

Yes. And yep, that's probably only useful for us.

I worry that in case of files - someone will have to go and check them periodically, and that would increase the reaction time.

Probably after #33 we should consider this point again. If we eventually add some monitoring system, then we can easily tune up Logs -> Monitoring -> Slack / E-mail / anything else.