Running the latest: indev-2 [Fixes: Music, .die startup lock - Adds: Optimize Program-O on LEMP with TLS, Quote Extensions, Remove(s): .rrc for high API abuse potential, UserPresence logging disabled for high API abuse potential -> frequent ratelimiting]

indev-2 is always the latest and tested to work. Default branch of my fork. This is for anyone who is interested. Despite `indev-2` it is tested well before updating.

`indev-2` safe / seamless updating (detailed version)

https://github.com/shikhir-arora/NadekoBot/tree/indev-2 the indev-2 branch is always up-to-date with NadekoBot/dev (official dev branch) with the stable voice updates in Kwoth#1139 .

To get the indev-2 tree, run the following in your home directory (where linuxAIO.sh is): (Linux)

(1) wget https://gist.githubusercontent.com/shikhir-arora/6b448ae2c07a890f08c422936f61256c/raw/60ecba4cfaaa7d1a386453d73fe8974068187560/mod_dev.sh && bash mod_dev.sh (you can split these up but the && will download and execute the script on Linux)

(2) This will update Nadeko to the indev-2 tree. (Just a modified version of the updater script), [pulling git clone -b indev-2 --recursive --depth 1 https://github.com/shikhir-arora/NadekoBot.git (the indev-2 branch)] [Don't worry about this second or third point below, just for explanation!]

(3) mod_dev.sh is the command linuxAIO.sh runs to update Nadeko to the latest development build. You can view it with the link above. The only difference is that this pulls my indev-2 branch instead of dev, but again they are kept up-to-date.

ONCE COMPLETE you can run bash linuxAIO.sh(Linux here, but should be similar for Windows) and simply start Nadeko (don't update it, just choose one of the startup options as usual) - the bot will start up as normal. Personally I use the auto-restart on .die without auto-update :-)

The below is an example of how to do it seamlessly: NadekoARN.sh is the exact same script that is called when you choose .die and auto-restart without auto-updating option. Once one runs the commands below, they only need to bash mod_dev.sh locally to update their bot with indev-2 as needed and, to start it up again, bash NadekoARN.sh (these are all parts of linuxAIO.sh

`indev-2` get - short version, just show me the commands! Just 4 commands:

tl;dr: cd to your main directory, get the two bash scripts, open a tmux session if needed and bash mod_dev.sh && NadekoARN.sh


vlexar@node1:~$ cd main/vlexar 

vlexar@node1:~/main/vlexar$ wget https://gist.githubusercontent.com/shikhir-arora/6b448ae2c07a890f08c422936f61256c/raw/60ecba4cfaaa7d1a386453d73fe8974068187560/mod_dev.sh && wget https://raw.githubusercontent.com/miraai/NadekoBot-BashScript/master/NadekoARN.sh && bash mod_dev.sh 

vlexar@node1:~/main/vlexar$ tmux new -s BotName

vlexar@node1:~/main/vlexar$ bash NadekoARN.sh

Differences Explained: (see commit log at bottom thread for overview)

Can always be viewed in full here: https://github.com/Kwoth/NadekoBot/compare/dev...shikhir-arora:indev-2 (indev-2 compared to dev branch)
Commands/Features added/appended: .liqukwd (list quotes matching a specific keyword, ordered by quote ID for easier searching and use with .qid <ID>), !!buffer or !!b (BotOwner only) to rebuffer music, and importantly: !!move or !!mv to seamlessly move Nadeko to any voice channel. This can also be used to refresh the queue, and fixes all hangs with the bot. see here for a video - watch in HD for best results
Commands/Feature updates cont'd: Hosted version of Program-O, the CleverBot that Nadeko uses currently. It is an AIML based system. The usage is the same, except it is customized with information for Nadeko, as well as a learning module. A crude version of it in action can be seen here: https://gist.github.com/shikhir-arora/2777e5b2ad7e254add4ec961864180e8 There are no changes to the major functionality, and the bot still uses the 75,000+ strings that the Program-O device uses. But, the benefits include - optimized, dedicated hosting on a VM alone, custom for this task, proper SSL/HTTPS configured with nginx over Apache for faster responsiveness.

LINK to the API with a test call: https://nadeko.link/nadekochat/chatbot/conversation_start.php?bot_id=1&say=Who%20designed%20you?

(it is managed under https://nadeko.link) I have messaged Kwoth the information for the admin panel as well several days ago.

Fixes: Fixes the music issues with Nadeko, including 95%+ of common issues and all major hangs. Allows seamless rebuffering and moving channels without loss of playlist information. Other fixes - Not allow the {prefix}die command to be issued as a startup command. This causes a severe loop and can crash the bot fairly easily.
Compliance Aims: 0648a2c47098e1af6224e51823194bc5b115c0d8 (30/04/2017) - removes .rrc rotate role colors - my comments are in the commit log below and in the commit description itself.
Compliance Aims: b4b88fa0a9381d197fc80a3459280fed72fcefa0 (10/05/2017) - disables UserPresence (online/offline/idle) logging for ratelimit and API respect - my comments are in the commit log below for 10 May 2017, and in the Why? section just a few lines down.

-Fork is Updated with the latest updates from dev and all command/response strings are kept up-to date. The dev branch here tracks the parent dev branch. The b2 branch tracks the parent 1.4 branch. All updates that are stable and tested go here.

Why? / Tasks [See Commit Log at the bottom for a quick table]

This is the version that I run, which solves many problems, aims to meet Discord API compliance for ratelimits, etc., and adds a few features:

[x] Fixes major music issues for 95%+ of use cases, add the proper move w/buffer [Standard Music Command] and force rebuffer [OwnerOnly] command ✅
[x] Fix bug to prevent {prefix}die from being used as a startup command ✅
[x] Add: .listquoteskeyword or .liqukwd to show a list similar to liqu but only for a certain keyword, ordered by the quote ID, which is approximately in the order of when they were added to the database. ✅
[x] Add: Custom, unique implementation of the same CleverBot Program-O running their latest development branch and support for things like AIML 2.0 (v2.6.5 as of this writing, ahead of the public API), with nginx over Apache (significant), its own VM, domain, and SSL for security, which should be pretty important as a primer ✅
[x] API/Ratelimit/Compliance Task (30/04/2017) - Removed .rrc for high potential of abuse, even with [BotOwner], see commit or log at the bottom of this page. ✅
[x] API/Ratelimit/Compliance Task (10/05/2017) - Disabled logging only UserPresence (online, offline, idle) updates, as these spew ratelimit issues all over the place. This is stopping the log capability from being used in multiple guilds effectively, which should be no issue, but the UserPresence updates means that there can only be one logged server, and even that is hit or miss depending on the size. Even with small servers, it is very easy to hit the ratelimit. From April 20-May 10, there were 30 cases of hitting the ratelimit, all specifically on the UserPresence within the LogCommand. on a single, small Discord server of just 40 users. An example: 23:47:25 LogCommand | Discord.Net.RateLimitedException: You are being rate limited. This has been well known for a while, and, besides making it a bottleneck for logging otherwise, this is a pretty big issue. On another note, I don't see the need to log UserPresence (online, idle, offline) - if such a thing is needed, a special app should be used to do so, not 'spamming' presence updates. ✅

Reasoning for Music

I think that is all right now. This post will be updated as necessary. I don't have the time to fully rewrite the music module right now as it needs some major changes which are better suited to a full rewrite, and the above fixes have been in place for weeks now here. 👍

I could explain my reasoning further with regards to Music, but it boils down to several things: a) it is not a stable thing to have a bot connected to voice channels for that long, both from a Discord API standpoint and from a stability/reasonable feature perspective especially when it is paused or has queued, stagnant music, and b) there are a lot of tiny changes with Discord.NET since the module was written, and the custom fork Nadeko uses may need some updates as we have a clearer way of audio handling now, and c) it is most important to move away from Google's API just because this is very slow, causes a lot of issues and limits several things. There are several sections that are using pretty antiquated techniques (many marked todo: from a long time ago) which cause a lot of overhead, unfortunately, so again this requires a rewrite over anything, which takes some time.

Feel free to shoot me any questions. Thanks!

Commit Changes and appropriate information/reasoning. (Updated 10 May 2017)

Date(s)	Commit(s)	Description
`26 March 2017`	529343ceb6469ebc25a9fdc3aab1796f2785126e	[Bugfix]: playlist bug where song would be added to the queue at the last index when repeatplaylist was enabled with repeatsong, as the ActionQueue events are not mutually independent
`19 March 2017 - 02 April 2017`	bf913030325444662446ee7c72c8b1c7b708e5c2 + 0c8fc55f00cfed570367914cdea3f7c6660ce11d + c635bdcd3c4a975e1066e43133ad15a817aef0c1	[Bugfix/Core Audio Fix]: modified `!!move` and related logic to fix the lock completely and disconnection timeout/no responses referenced. Added buffer command for `[OwnerOnly]` to manually initiate a rebuffer if needed. Care taken to ensure that no playlist metadata (position, playlist tracks, etc.) are lost, even after heavy testing. This closes most issues from `Kwoth/NadekoBot#1053` and the realistic next steps would be a full rewrite of that module, which is a large task to refactor.
`05 April 2017`	d00ec1e51cf7b27afef1caa19d5ec7b078e9ce83	Added: CleverBot Program-O (same as parent), on LEMP stack with https vs. slower public API on an Apache system with no TLS
`10 April 2017`	10170e6f9c4af826863c2ffa31d4c8c93d259777 + cbc2599b6fec03e8045a8a921d189a244148a6a8 + ef6dc339199742a0b6b955cf9a52d7fa113c94e9	Added: `.listquoteskwd` command to supplement the `.quoteid` to search by quote ID. Some users have 100's of pages of quotes, so it makes sense to be able to filter the quote ID list for a specific keyword (which are searchable from the `.quoteid` feature I added and was pushed to the parent/dev branch)
`11 April 2017`	d570992f8adf43aa7370e050a6f5f38a766d5111	[Bugfix/Lock]: `.die` on startup by ignoring that as a valid startup command. `.die` (auto-restarts bot, with/without updates depending on your config) will cause the bot to go into a deadlock and loop until either it gets ratelimited and disconnects from the gateway, user catches it and can terminate the process, or until SIGINT is sent to kill signal
`30 April 2017`	0648a2c47098e1af6224e51823194bc5b115c0d8	[API/Compliance]: Disables `.rrc` commands. My comments on the commit: "Yes, these are [OwnerOnly] issued, if someone uses them responsibly it is fine (with maybe their own user role with 1-2 people), but I can see there's too much spam potential here, which counts more than functionality, especially when a role can have thousands of users updating that many will certainly trigger ratelimits, and not a great thing to have that option there for those who don't care about the ratelimits."
`10 May 2017`	b4b88fa0a9381d197fc80a3459280fed72fcefa0	[API/Compliance]: Disabled logging only `UserPresence` (online, offline, idle) updates, as these spew ratelimit issues all over the place. This is stopping the log capability from being used in multiple guilds effectively, which should be no issue, but the UserPresence updates means that there can only be one logged server, and even that is hit or miss depending on the size. Even with small servers, it is very easy to hit the ratelimit. From April 20-May 10, there were 30 cases of hitting the ratelimit, all specifically on the UserPresence within the LogCommand. on a single, small Discord server of *just 40 users.* An example: 23:47:25 LogCommand Discord.Net.RateLimitedException: You are being rate limited. This has been well known for a while, and, besides making it a bottleneck for logging otherwise, this is a pretty big issue. On another note, I don't see the need to log UserPresence (online, idle, offline) - if such a thing is needed, a special app should be used to do so, not 'spamming' presence updates.

shikhir-arora / NadekoBot