MarcusOtter / discord-needle

Needle is a Discord bot that creates Discord threads automatically.
https://needle.gg
GNU Affero General Public License v3.0
200 stars 53 forks source link

💡 Auto-thread prerequisites #93

Open n1ckoates opened 2 years ago

n1ckoates commented 2 years ago

Describe the improvement

A way to select requirements that a message must pass for it to be auto-threaded.

For example, a server might want to only auto-thread on messages that contain a link, or have a keyword, or are from a specific user or bot (#61).

I'm not exactly sure how this should be implemented. The string option only allows selecting one choice, and this would probably work best as a dropdown. We might be able to use modals or a dropdown message component.

Problems this improvement solves

In one of the Discord servers I'm in, we have a news channel, where members post links to news articles. Needle is setup to auto-thread messages in this channel. However, other messages are sometimes sent in this channel, so it'd be nice to only auto-thread on messages that post an article.

Alternative solutions

No response

n1ckoates commented 2 years ago

After thinking about this a bit more, I have an idea for the command structure.

/configure requirements would be the base sub-command.

A message would only be auto-threaded if all of the requirements passed, not just any.

luniks commented 2 years ago

Nice feature 👍

It would be great to have also a number of answer as prerequisites! Discord already suggest to create a thread when a chain of 3 replies occurred : image

MarcusOtter commented 2 years ago

@luniks I think this feature would just apply to channels with auto-threading enabled. Those channels shouldn't have a chain of replies as each message would create a new thread.

luniks commented 2 years ago

Hello @MarcusOtter!

Thanks for your feedback. Perhaps I misunderstood the topic of this issue. My guest was having the possibility to add conditional rules on how the auto-thread process is triggered (for exemple : a certain role of a user).

So the basic idea is to create a new thread only on specific use cases, and a chain of replies could be one of them for me. For sure with this configuration, the auto-threading would become the exception and not the usual case ;)

But maybe I missed something or their is some technical constraints I am not aware of. Please let me known!

MarcusOtter commented 2 years ago

Right, I think I was distracted when I read your original message and misunderstood. Sorry about that! Amount of replies definitely makes sense. Might be a bit tricky to figure out implementation-wise but shouldn't be impossible :)

MarcusOtter commented 2 years ago

We could make this feel more like Discord's search feature where you can have a dropdown with options for fields like this (and the ones you mentioned previously): bild

This would make it easier for people to select which sort of attachment they want to require. Discord's search function allows to filter by link, embed, file, image, sound, video, and sticker. Not sure if all of those are programmatically easy to determine but just throwing the idea out there :)

Maikel1990 commented 2 years ago

Any update on this? Would be very nice if this is implented for a suggestion channel for example.

goodideagiver commented 2 years ago

Consider adding option to block discord respond feature.

In this scenario: obraz

The bot could delete responding message because user should respond in a thread.

MarcusOtter commented 2 years ago

I think we should make it configurable on what actually happens when the prerequisites are not fulfilled, something like

Snailedlt commented 2 years ago

@nchristopher great idea, and lots of neat arguments too! To add to the arguments we could have

MarcusOtter commented 2 years ago

A regex option would be cool - but we need to make sure that we do it safely (there are regexes that can be constructed to be very very slow or hang the program).

Snailedlt commented 2 years ago

This might be useful then: https://www.npmjs.com/package/safe-regex Never used it before, but seems legit with 19,516,070 weekly downloads

YozoraXCII commented 2 years ago

To follow up on this, the ability to exclude some of these criteria would make this feature a lot more powerful.

As an example, if I have Bot1, Bot2, Bot3 etc up to Bot10 and I want to exclude Bot10 from creating a thread, it is much simpler to say "user is not Bot10" than it is to say "User is Bot1 or User is Bot2 or User is bot3 etc through to Bot9".

Snailedlt commented 2 years ago

@YozoraXCII yes, this 100%!

Just adding a not operator should be fine, that way you can have a list as an input. Having AND and OR operators too would make the solution a lot more powerful.

For example: user: (@12345678 AND NOT @1245599283) OR @8489185772`

AND, OR and NOT can also be replaced by symbols if you want.

AND = && OR = || NOT = !

MarcusOtter commented 2 years ago

The implementation would likely be something like two parameters includeUsers and excludeUsers that can take a role or user mention. If there is overlap between them, the exclude should "win". That way we don't force normal users to write boolean logic in a raw string without any help, and instead take advantage of the built-in parameter types in Discord which gives them a dropdown. This makes for better UX:

image

Snailedlt commented 2 years ago

It is more verbose for sure, but much less powerful. Could we also get an alternative implementation with Boolean expressions supported?

MarcusOtter commented 2 years ago

I think the user experience for the average person is more important than the very very high configurability for the more technical-minded software people. When we start adding boolean logic and scripting to the bot it's better to just let people fork it and run their own versions of commands. So I don't see myself adding something like that, sorry.

EDIT: Not to mention how much more complex some sort of scripting logic would be to implement, and a key design goal of this bot is to keep it as simple as possible.

Snailedlt commented 2 years ago

Okey, fair point :) Guess we can always fork if we want more advanced functionality

auwsom commented 1 year ago

@MarcusOtter ( @luniks ) If I want to fork this to add Auto-thread option after X messages, could you point me in the right direction?..

I assume the settings are stored in this file (for example includebots): https://github.com/MarcusOtter/discord-needle/blob/ee1e7174fcca94aa36d29b319883157360180dac/src/models/AutothreadChannelConfig.ts#L28

And created by this file: https://github.com/MarcusOtter/discord-needle/blob/ee1e7174fcca94aa36d29b319883157360180dac/src/commands/auto-thread.ts#L207

And then the thread creation is checked with "shouldHaveThread": https://github.com/MarcusOtter/discord-needle/blob/ee1e7174fcca94aa36d29b319883157360180dac/src/services/ThreadCreationService.ts#L58

I just noticed the comment "// In the future if we have prerequisites we need to check those here", so I assume this is where you imagined putting this.. https://github.com/MarcusOtter/discord-needle/blob/ee1e7174fcca94aa36d29b319883157360180dac/src/eventListeners/ready.ts#L73

So I would need some logic in a listener like "if latestTenMessages has only 2 users, then allow thread creation"?

Thanks for any input you have. Cheers

PS: I'm leaning toward a thread being created only when a "reply" is used. This should simplify everything.

auwsom commented 1 year ago

I think all I need to change is above if (!lastMessage) continue; add if (lastMessage != REPLY) continue;.

I'm not sure yet how to check if lastMessage is a reply.

According to https://stackoverflow.com/questions/67032774/detect-if-message-is-a-reply, the MessageType can be checked for Reply. https://discord-api-types.dev/api/discord-api-types-v10/enum/MessageType#Reply

I think it should be: if (lastMessage.type != "Reply") continue;

https://old.discordjs.dev/#/docs/discord.js/main/class/Message?scrollTo=type https://github.com/discordjs/discord-api-types/blob/main/payloads/v10/channel.ts#L698

auwsom commented 1 year ago

Okay, I've forked and made a single line patch from my last message. Testing now. https://github.com/auwsom/discord-needle-reply

Parking a couple of feature requests from Discord Support. Apparently, this is how Slack works by default. https://support.discord.com/hc/en-us/community/posts/4405451136535-Creating-threads https://support.discord.com/hc/en-us/community/posts/4405186575639-Auto-create-threads-in-announcement-channels

auwsom commented 1 year ago

Sweet, it works! @luniks https://github.com/auwsom/discord-needle-reply.git

Only two lines were changed. In discord-needle/src/services/ThreadCreationService.ts, import { MessageType } from "discord.js"; was added on line 35, and if (message.type != MessageType.Reply) return false; was added on line 69.

auwsom commented 1 year ago

@MarcusOtter Hey Marcus, I'm going to add this issue as a solution to the Discord Support feature request pages above. Slack is set up this way by default, and it really cuts down the clutter. It would be great to add this to your repo, so people don't have to use my forked version, and can use your website bot invite.

Thanks for making this repo! This feature is a big deal for me using Discord and keeping it organized.

Like this comment if you want to see this in Marcus's Needle.

auwsom commented 1 year ago

You can use this bot I'm hosting for the time-being. Or you can host your own in GCP: https://discord.com/api/oauth2/authorize?client_id=1110757452032520264&permissions=326417517568&scope=bot

btw, the bot command to turn it on is /auto-thread on and needs to be done per channel, afaik. also , the GCP Debian and Ubuntu images have old version 12 of nodejs. Install 18 with curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash - &&sudo apt-get install -y nodejs. It comes with Npm.

MarcusOtter commented 1 year ago

@auwsom Thanks for looking into some of this stuff. Your version seems to make sure that Needle never creates a thread if it is a reply, which is also an interesting aspect we might consider for this prerequisites feature in the future. It will not be merged into the main repo as-is because that would be a big breaking change for everyone and is also not supposed to be the default behavior - it needs to be configurable. But the idea is there, so thank you for showing that it is possible!

auwsom commented 1 year ago

@MarcusOtter sure, no problem.. just to be clear, it makes a thread only when the reply is used. This is how Slack is by default. Maybe we can make it an option.

Junebugging commented 1 year ago

I think the user experience for the average person is more important than the very very high configurability for the more technical-minded software people. When we start adding boolean logic and scripting to the bot it's better to just let people fork it and run their own versions of commands. So I don't see myself adding something like that, sorry.

Agree with this, and this issue has been up for over a year now, it'd be far better to have a rudamentory version of this feature soon, than to wait another year to implement a more complex version of it.

Junebugging commented 1 year ago

If someone could point me to where in code the messages are intercepted and read, I could create a branch and try to begin implementing this

auwsom commented 1 year ago

@Junebugging check out my code (and some details in my comments above) I'll repost the link to my fork here in a second.. https://github.com/auwsom/discord-needle-reply

One thing i wanted to check when using it in Discord is that it creates "a subtopic" for each thread in the left side bar. I'm not sure how Slack handles that.

MarcusOtter commented 1 year ago

I appreciate the enthusiasm here and I recognize that this is the most commonly requested feature by far. I don't have a lot of time to work on this bot so that's why progress has been lacking.

For me, the main hurdle right now is the UX. What would configuration of this look like? It needs to be very generalizable and easy for us to add more requirements down the line. And it needs to be very clear to the user how it works. The best way would be if we could configuration of this to /auto-thread in some way (since that's how all other settings of auto-threading behavior is handled), but I think the amount of parameters would be very overwhelming (and I think there's even a hard limit of how many parameters we can have for one command).

Some things we still need to figure out:

I don't have good answers for these

Junebugging commented 1 year ago

The most bare-bones version of this feature is 100x better than none at all, I believe. Such advanced requirements might only be used by a very small minority of the users, and I don't think such advanced features are currently used by anyone, as I can't find an alternative bot that does this.

But if you don't want to add the feature unless it's got this modular and future proof functionality, maybe you could handle it through letting the user repeat the option, so say a requirement argument. If you could then somehow suggest or show hints to the user, auto-complete suggestions would work, then the user could input either "+" or "-" followed by the requirement, all the + requirements needing to be met, and negations always taking priority. + = Include requirement - = Exclude ! = Override

Example of usage:

The bot would start with the message that was sent in the configured channel, then it would pass it through these include-requirements, then excludes, then overrides, in that order. If it passed all these requirements, a thread is given to it.

This is a solid filtering method that I think would fit even a majority of advanced users' requirements, while in my opinion being intuitive enough for most people to understand without any prior knowledge, simply by the prompt alone. The filtering options this would provide I believe is more than enough, because the current alternative is having no requirement options at all.

This styling would follow Discord's already established style of filtering syntax, which the target demographic of server-managers are definitely familiar with, only with additional options. image image

It could also prioritize requirements in the same order Discord does, based on specificity, which would be intuitive to server managers.

(However copying discord's permission system directly of course wouldn't work, as users permissions aren't AND'ed together based on all their roles. Users + perms are OR'ed together, and then overriden with - perms of a higher specificity. Aka roles first based on role order, then category, then channel, then channel rules for that specific user. This system won't work for the bot, as it's a completely different use-case. Just wanted to clarify that.)


Processing

I believe the most logical functionality for the requirement tag, is that each step needs to be true for the message to get through to the end and be given a thread, hence the name requirement. This is essentially AND'ing the requirements, which I believe will be the best fit in most situations I can think of for needing the auto-thread functionality. However for OR'ing, the later on ! suggestion can help them.

When a message is received, it must meet every + requirement:

And afterwards, is denied if it meets a - requirement

Then, ! overrides are processed in the same order. If an override would include a message that was previously filtered out, it will be included again. Because of this, it might be best to process overrides first, and then skip the non-override requirements if the override includes or excludes the message.

In addition, if a user:@person requirement has been set, this overrides everything, because it's considered the highest specificity. Discord does permissions in this way too, with more specific requirements being prioritized, such as perms for specific users for specific channels overriding every other perm.

Example

Say as an example you want images sent by people with the @artist role to be threaded, but you want them to be verified because the @artist role is self-selectable, you'd pass requirement:+role:@artist requirement:+has:image requirement:-role:@unverified The bot would then read every incoming message, and if it doesn't fit either of the first requirements, the bot stops evaluating.

Overrides

If you want even more advanced filtering to be possible, a somewhat intuitive option would be to tell the user that placing a ! before + or - would make it take priority over everything else (!- taking priority over !+). I believe this would fulfill even more of the advanced users requirements, to the point where I'd recon that there'd be barely any people who wouldn't have their needs met by the feature somehow. And either way if some people still want more advanced features, they would likely be happier to have this solution which lets them meet some of their requirements, rather than having none of their requirements met at all.

The requirements could alternatively be overridden based on the order the requirements are written.


Alternative approaches:

(All of these suggestion does however depend on how Discord bots can present information to the user - but even if this is limited, most people looking for a bot with such a specific thread-feature as this would probably be willing to go to the webpage to quickly learn the format of a simple repeated string-based requirements system like the one I've suggested, especially if the bot can give a hint pointing to the help website or command.)


An imperfect but decent solution is better than no solution in my opinion, but I absolutely see and respect your dedication to wanting to plan out how you implement things, to make sure you don't add features that are a pain to work with or change in the future. It's an important thing to prioritize, but personally at least. I believe that if a great deal of time has passed and no better solution has presented itself, it's unlikely that a better solution will ever appear. At that point I believe it's best to take an imperfect implementation, when that imperfect feature can be of great help to people even in its imperfect state. I don't think there's a much better way to implement this either way, unless the entire input system is re-built to use reactions and dropdowns like some other bots do, or unless Discord bots can present more advanced options or popups to users, within a simple message-command, than I'm aware of.

PS: Sorry for the ridiculously long message, hope I made my suggestions clear despite the length, and that they can be of use somehow.

MarcusOtter commented 1 year ago

Lots of good points, thank you for all of this. It's super helpful to go back and forth a bit on this and to write it out as most of it has just been in my head 😅

The most bare-bones version of this feature is 100x better than none at all

I agree. At the same time, we don't want to come up with a patch later that completely rewrites how the bare-bones version worked, that will just be frustrating for users. We still need to plan ahead and make sure it can be expanded on, unfortunately. That doesn't mean we need to solve the complex and highly unique case that I mentioned as an example, but here are some use cases I think we actually need before shipping a bare-bones version (which has been requested numerous times):

maybe you could handle it through letting the user repeat the option

As far as I know, this is not possible in Discord. A slash command can only receive a set number of arguments and arguments can not be repeated.

If you could then somehow suggest or show hints to the user, auto-complete suggestions would work [...] All of these suggestion does however depend on how Discord bots can present information to the user

Yes, I think this is actually the biggest obstacle right now. It's really hard to display information like this intuitively to users and that's kind of where I get stuck, in the specific details of implementation (what should be a command, what should be an argument, etc).

I think the suggestion you put on the table is okay UX wise, but the auto-complete interface really does not work well in my opinion (and also, is not very commonly used by bots, so many users will likely not understand how it works). Especially pinging specific roles and channels would be difficult in a setup like this because, yes, we can give suggestions, but sometimes the names aren't unique. For example you can have multiple "#support" channels under different categories, but a user could never figure out which one is which in the auto-complete. We would need Discord's channel picker here, since it shows the category the channel is under.

EDIT: Of course the channel filter makes no sense here, but a similar problem could occur with roles or users with nicknames. The main point is that Discord's dropdowns are easier to user for normal people than selecting (or not selecting) a text suggestion.

Another benefit of using the actual Discord-native dropdowns would be that we can give validation before they even send the comamnd (e.g. not a valid channel, not a valid user, etc). With auto-complete and raw text, a user could send whatever they want, and our bot would have to try and parse that string, and only then get a reply back, which is a more frustrating experience. Especially if you've written multiple requirements already. The trial and error loop becomes much more cumbersome and I think I would have to dedicate a lot more time to helping users set up Needle to work for their servers (which I am basically not doing at all right now! only RegEx help every now and then).

Let users pass a string with multiple requirements with AND & OR support in a single requirement if they want to

I still stand behind what I said last year regarding writing boolean logic or raw text - it is not a good user experience, and 99% of users don't want it. The 1% that want the advanced functionality can fork the bot and implement their own logic. I don't want to encumber/overwhelm "normal" users just to make the bot more advanced, even if that just means a handful of new parameters in a command. It would be great if we could just use the built-in Discord dropdowns and pickers over raw text. Not to mention how convoluted and overly complicated the implementation would be.

I believe that if a great deal of time has passed and no better solution has presented itself, it's unlikely that a better solution will ever appear.

One thing that would actually just make this so much better is if Discord just released multi-page modals and added more components to modals (like user/role dropdowns, channel picker, etc). They've been talking about it for years but developer experience isn't exactly their main priority, unfortunately. Just thought I'd throw that out there - it's possible that Discord does actually release things that make the UX so much better. But you're right that sitting and waiting around for something better to show up isn't a good idea.

MarcusOtter commented 1 year ago

Here's where my head is at right now:

At least we could do the use-cases I mentioned in the last message with this. I'm thinking the include- parameters would be AND:ed together (all of them need to be fulfilled), and an exclude- parameter always wins over the includes.

The good thing about this UX is that we use Discord's native pickers for everything and we can give the users good hints throughout the process. The drawback is that it's a separate command and is a bit verbose, but I'm telling myself that this makes it more clear for the users.


Here's another suggestion, splitting include and exclude into two commands, to make it even more obvious how it should be used, and less verbose:

One drawback of this is that it's not obvious that setting /allow content: image would also mean to block everything that is not an image. So maybe we would need commands like this too, with this:

OR

Junebugging commented 1 year ago

Oh yeah that's true, if Discord's selector prompts can't work within a text command like that that complicates things a bunch.

And I completely agree that requiring the user to learn to create a complex filtering string is not desirable at all, especially when Discord selector prompts don't work, something that could cause same-name conflicts. I'm still of the opinion that a difficult-to-use feature is better than none, as long as it can be invisible to most users and as long as it is replace-able with a better feature in the future - so with that in mind it could possibly be considered an intermediate not-suggested-by-default filtering option for advanced users and dedicated server managers. But - as long as it's possible to implement a better version of the feature, within a reasonable timeframe, it's much better to let the users wait a bit, to introduce the feature at a point where it's more "complete".

(You don't want most users to get used to a feature that will be completely changed in the future. That's why the only intermediate alternative I suggest, if this feature is to be implemented properly, is an advanced string-only option which would remain supported in the future. This would be because it ought to be seen as simply a way to "bypass" passing each argument individually for power-users, but it would use the same argument formatting. If it takes more time to implement the parsing of such a string than it takes to implement the feature properly then that's absolutely useless though. )

But again, the intent of suggesting it is that it could be an easier-to-add advanced option for the users who need, it in case the feature doesn't get implemented properly, but that could remain alongside a proper implementation. I don't think this is necessary at all, so long as the feature is likely to be implemented in some other way.

So for now, not something to think about.


Anyways, back to the proper implementation

I really like your filter suggestion! I think it makes the most sense and is the most intuitive for AND'ing together filtering options, as long as it's clear that it interacts with the auto-thread command.

Though it still isn't entirely clear that they'll be AND'ed together, so what about your suggestion (either first or second), but an additional option for AND'ing filtering options, called require? Both include and allow imply that the options will be included or allowed, OR'ed together - but having the option to OR filters can have it's uses. So what about including the include or allow as the OR option, and then adding require, which intuitively tells the user that "this requirement MUST be met" for a thread to be created.

The including, requiring and excluding could be handled by:

  1. If no include arguments, start with include = all
  2. Check if message matches any include option
  3. Check if message matches every requirement
  4. Check if message matches no exclusions
MarcusOtter commented 1 year ago

Require is a better word for it, thank you. With /require and /block I think it's clear without the /... all commands. Maybe we should have a mode parameter or something so you can choose for yourself if you want to OR them or AND them, which still works with require (require all of them or require at least one of them), while also making it pretty clear that it is AND by default.

I think /block is a bit harsh too (kinda sounds like Needle will take a moderator action), perhaps /skip or /exclude is better. Or maybe /ignore is even better, because Needle will do just that?

Also maybe to make it clear that /require and /ignore are related, they should be named /filter require and /filter ignore. I think we're starting to get somewhere with this :)

MarcusOtter commented 1 year ago

We also need to figure out what to do with the current include-bots option since that would fit much better in a filter. I think it makes more sense if we by default set that to true when this is implemented which would be a breaking change, but for this big of a feature I don't mind a breaking change

So by default we also create threads on bots, and then we remove the include-bots option. People can do /filter ignore users: @bots on a bot role if they want to exclude it

Junebugging commented 1 year ago

I don't think user complexity should be a concern when other more basic features, like custom titles, have regex like this without an explanation:

image

And all of these options

image

To configure Needle properly you already need to be a bit technical, so needing a bit of technical knowledge for this kind of feature wouldn't be deviating from this

Junebugging commented 1 year ago

I like your suggestion, something akin to

But the command can be changed later on

EngineExt commented 1 week ago

Any update on this? To start with, why not add an option to include or exclude the role(or ID user/bot) that you need? That would probably solve a lot of people's problems.