The first absolutely gigantic rewrite

[raven0034]

What has changed:

• The project has gone through an initial restructuring to allow better grouping and navigation of cogs.

• In doing this restructure, previously used things such as credentials.csv have been scrapped in favour of a central config.json, which allows storing the token, database url, global prefix and a tree of cogs to be loaded (previously this was hardcoded)
• utils was previously imported into every cog, making some aspects of reusability difficult (and also inefficient), hence utils is now imported through the main file which cogs can now access. As of later on in the development of this PR, utils was split into its original file and a cog, where the cog was made to be required. This allows utils to start making use of some listeners, although a neater solution would be better
• There are now required cogs and optional cogs. Required cogs are config and tasks as these are considered "core, required features" (half the bot would break if config wasn't loaded). Optional cogs can now be added or removed, and to load them simply requires adding to the tree in config.json
• Todos (renamed to tasks) has been moved into its own cog, although I have plans to further rewrite this to allow cogs to register types of tasks in some way.
• adambot.py now contains the AdamBot object only, startup steps etc have been separated out, making start.py (handily named) the entry point for the project now. Some of the processes like handling missing dependencies and loading cogs have been abstracted out to /scripts/utils to keep adambot.py relatively clean.

Some new cogs have been added:

• The trivia cog has been rewritten, with the vast majority of previous bugs (e.g. with multiple members answering and the question count freaking out) fixed
• A new starboard cog has been added (self-explanatory). Allows custom emojis to be used as well but I guess starboard sounds better than emojiboard
• Introduced a bot cog, which will host more features directly related to the bot rather than having them dumped in the member cog
• A new actions cog has been added. Actions are per guild groups of commands that can be given arguments within particular contexts. Outputs can be added to these actions where appropriate. These are by and large hugely more convenient (for example, you can use them to create command aliases, or you can use them to simplify adding a group of roles to a user) for expandable per-guild custom content than previously where some server-specific code was in place for particular functionality. Whilst there are ways this cog can be improved upon (as suggested in the todos section at the top of its file), it is now in a functional state.

discord.py

• Updated the discord.py requirement to V2. This refers to the state that the master branch of the original repository was left in at the time it was discontinued. So far in testing this version seems to be in a stable state. Updating this requirement allowed some of the actions code to exist, as V2 has a handy method run_converters, which gives us a method version of what the library does under the hood when a command is invoked. Avatars, icons etc have now all become assets so there has been some minor tweaks to get those working again. All discord.py objects that work with datetime are timezone-aware in v2 so there was also some minor tweaks to get localisation working again.
• Whilst discord.py is now not being maintained, there are no limitations that I have encountered which need to be immediately resolved, however I do accept that we will need to look for a well-maintained fork in the future.
• Update: 4/1/22 - We have now switched to an active fork known as Pycord. Some relatively minor changes (e.g. discord.errors -> discord) have been made to get the bot working again.

Dependencies (update: 4/12/21)

• In an effort to make the bot easier to set-up, the bot now verifies whether all the dependencies are installed
• This includes version checks where applicable.
• Missing dependencies are now installed automatically - pip, setuptools and wheel are optionally upgraded to the latest version to save any issues with dependencies not building (e.g. matplotlib doesn't build on Python 3.10 without the latest version of pip & setuptools which... doesn't come with Python 3.10 lol)

Other somewhat notable changes

• Python version has been bumped to 3.9.7 (have also now begun 3.10 testing. As of 27/12/2021 we are now making small use of some Python 3.10 features, including the new representation of Union type annotations and match-cases. As such the project will now require Python 3.10).
• Added CodeQL
• Several general bug fixes, code reformatting etc has been done along the way
• Most if not all functions are mostly type annotated, both for arguments and return types
• The way config keys are accessed has been changed slightly to mostly eliminate bugs that occur when the config hasn't yet been loaded (i.e. at startup)
• is_staff and is_dev have now been moved to decorators so that there aren't daft duplicate checks everywhere. The original is_staff functions remains in the config cog so that it can be used for other purposes than as a command invoke check
• The vast majority of the code relating to the GCSE server has now been removed since it is redundant

Is this everything?

• I've probably forgotten something that's changed in this batch of commits
• Further rewrites, restructures etc are planned for the future, but this is a point at which I can say I'm happy with the progress made for now

[raven0034]

Should probably add some stuff for not storing keys in actions where the values are empty for the sake of saving some DB space (e.g. if you're just storing a key's value as an empty list there's no point storing it in the first place)

[raven0034]

The starboard cog needs tweaking somewhat so there's not massive overhead every time there's a reaction with it requesting starboard entries. Also could do with some better way of tracking entries, specifically whether they need to be created or not, since spamming/lots of people reacting quickly can cause the bot to have a fit and create two messages. Some sort of queue or something similar may be ideal so it doesn't conflict.

[raven0034]

Will pick up work again on this at some point in the near future. Want to transition to Python 3.10 and make proper use of some of the new features.

[raven0034]

On windows: matplotlib 3.4 will not build on Python 3.10, but 3.5.0rc1 does appear to work

[raven0034]

Next steps: decentralise the db schemas and config keys (not decided on whether to do this in separate tables or feed into the config table as now) - this is part of an effort to ensure changes made do not have an impact on cogs they aren't related to. Eventually a cog should be able to be "dropped" into the project and (obvs providing it conforms to certain protocol specific to Adam-Bot) work without having to modify unrelated files within the project itself.

For stuff like the GCSE server for example a repo could be created with cog(s) specific to that server without being part of the "main project" ie other people can make use of it without extra baggage.

[raven0034]

Finally made it such that cogs largely have control over the config keys that they make use of - they are now stored within each cog's config file. Some keys that are key to the bot actually functioning have been retained in the old format (namely the staff role, logging and the prefix). This leads us closer to the "plug and play" style of cogs where they can simply be added to the downloaded instance of the bot, added to the config file and can be made use of without having to modify half of the project files in the process.

[raven0034]

Given that config key creation is a lot more dynamic there is also now extra code to handle creating new columns on the spot although ONLY if there is data to be propagated for such a new column for any given guild (ie it will NOT create columns where all the data will just be null). With that said it may be worth looking at removing columns that have become all null for pre-existing columns, but this is of a lower priority.

[raven0034]

Extending from this, as such config keys are only loaded on where the cog has been loaded in - although at some point I will look at clearing up phantom keys (ie keys that are loaded but then a cog fails to load (hopefully not an issue that will occur with the current cogs, but something that should be accounted for)

[raven0034]

Just spotted "bruhs" are currently broken because it's not actually a config key

[raven0034]

This is a step forward considering before you'd have to manually interact with each and every db you had one/more instances of the bot connected to add the new columns - instead now just add to the cog config and it gets handled upon redeployment. Ideally we could also do with validating the columns have been created as expected (ie with the correct type(s) and any constraints)

[raven0034]

🙏 let's do it