etkecc / baibot

🤖 A Matrix bot for using diffent capabilities (text-generation, text-to-speech, speech-to-text, image-generation, etc.) of AI / Large Language Models (OpenAI, Anthropic, etc.)
GNU Affero General Public License v3.0
39 stars 3 forks source link

Baibot reacts not only for !command and baibot, but also for any string contain baibot #21

Open CybotTM opened 1 week ago

CybotTM commented 1 week ago

Baibot reacts not only for !command and @baibot, but also for any string containing "baibot"

Is this intentionally and configurable?

for example pasting URLs like https://github.com/etkecc/baibot/issues in the chat room triggers baibot.

May not happen that often for "baibot" but if "accidently" use a more common string as bot name, like "bot" or "ai", it may trigger the bot far too often.

spantaleev commented 1 week ago

This depends on the Matrix client that is used.

baibot tries to determine if it's been mentioned by inspecting the m.mentions field of message payload (as per the user mentions spec).

If you send a message like this:

{
  "type": "m.room.message",
  "content": {
    "msgtype": "m.text",
    "body": "https://github.com/etkecc/baibot/issues",
    "m.mentions": {}
  }
}

.. which contains an m.mentions field, baibot can clearly see that no one was mentioned in this message and it's safe to ignore it.

The spec says that:

To avoid unintentional notifications, it is recommended that clients include a m.mentions property on each event. (If there are no mentions to include it can be an empty object.)


Older Matrix clients (which do not support the user mentions spec at all) do not include a m.mentions field at all.

They would send a message that looks like this:

{
  "type": "m.room.message",
  "content": {
    "msgtype": "m.text",
    "body": "https://github.com/etkecc/baibot/issues"
  }
}

In such cases, baibot does not know if the message aims to mention it or not, and it falls back to doing string-matching ("is my full user id (e.g. @baibot:example.com) or display name (usually baibot) contained in the message body?").

One such client is Element iOS. Perhaps the old Element Android is behaving similarly.

We'd love to drop this string-matching fallback and only rely on proper mentions (as supported by newer clients), but.. old clients are still very common. Without this string-matching fallback, old clients would not be able to mention the bot at all.

Perhaps the string matching logic could be made smarter, but that's difficult too. The way people mention the bot varies:

Trying to improve string-matching should somehow account for all these use cases and will likely break some of them. I don't know if it's worth messing with it.


Alternatively, we can try to strip certain well-known false-positive-triggering sequences (e.g. https://github.com/etkecc/baibot; others?) from the message body before running our current naive string-matching logic. This should improve the situation somewhat without breaking anything.


Another potential solution is to wait for the issue to auto-resolve itself as people migrate to better Matrix clients (like Element X, Fluffychat, ..).


For completeness, I'll also mention a workaround that anyone can apply right now.

As mentioned above, the bot does string matching against its full Matrix ID (e.g. @baibot:example.com) and against its display name (usually baibot). The problematic behavior occurs with the display name (baibot).

This is configurable, however, via the user.name configuration setting. If you were to change this to something else and restart, the bot would change its display name in all rooms, and you can avoid triggering it accidentally with the word baibot.

CybotTM commented 2 days ago

We are using latest Element Web and Desktop Client. (It seems Element X does not have a desktop client and is lacking support for spaces and threads)

About the m.mention, you are right (not that I thought you are not right, I meant I just checked it to see it for myself), Element did not add m.mentions to the event.

spantaleev commented 1 day ago

Element Web/Desktop should be adding a m.mentions field.

Only old Element on iOS and Android lacks support for it.


Element X supports threads in the sense that it can display threaded messages and you can long-press -> "Reply in thread" to continue the thread. However, it doesn't render threads like old Element did (having a separate "thread view").

I find the new behavior good enough for usage with baibot. If you use many threads at the same time, it can get confusing, but.. I personally don't use it that way.