thomst08 / requestrr

Requestrr is a chatbot used to simplify using services like Sonarr/Radarr/Lidarr/Ombi/Overseerr via the use of chat. Current platform is Discord only, but the bot was built around the ideology of quick adaptation for new features as well as new platforms.
MIT License
206 stars 16 forks source link

“The application did not respond” - error within discord #9

Open andrewmcd7 opened 7 months ago

andrewmcd7 commented 7 months ago

Hey, thanks for all your time and effort on this bot. It’s great when it works however we are getting almost daily “the application did not respond” within discord.

We manage a few discord servers. So we have an about 5 requestrr containers.. actually 10… 5 for regular requests and 5 for foreign requests that get added to different arr’s.

We split the containers we use between all of the discord channels to split the load. But a lot lately we are getting this error. We dont know if it’s a requestrr error or an issue with discord api. A lot of the time we either have to reboot the container (sometimes is fixes it), redeploy the container, and if that doesn’t work kick the bot and readd it to discord. It was manageable at first but now we are getting it more and more.

Im hoping we can work together to resolve this issue. IMG_6565

thomst08 commented 7 months ago

Hey @andrewmcd7,

That sounds like an interesting setup, this might be abit tricky.

You could be right, it might be an issue with the discord api, but the only way we can find out is check the logs of each container when they error and capture why it errored. Can you collect a copy of the logs (in text form) and post them? That way it should give an idea why it's happening.

If your unsure how to get the logs, this link might help: https://github.com/thomst08/requestrr/wiki/Where-are-the-logs%3F

Let me know how you go. :)

andrewmcd7 commented 7 months ago

I will next error. Thank you

andrewmcd7 commented 7 months ago

Hope this is helpful.

warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot has been restarted. warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Removing movie notification for user with ID 979274836897595462 as it could not be found in any of the guilds. [2024-02-10 20:27:05 +02:00] [111 /ConnectionCl] [Crit ] Connection terminated (4000, ''), reconnecting [2024-02-10 20:44:19 +02:00] [111 /ConnectionCl] [Crit ] Connection terminated (4000, ''), reconnecting [2024-02-10 22:25:35 +02:00] [105 /WebSocketRec] [Warn ] Unknown event: VOICE_CHANNEL_STATUS_UPDATE payload: "status": null, "id": "954093177139253318", "guild_id": "953038263797563422" [2024-02-11 00:00:56 +02:00] [111 /ConnectionCl] [Crit ] Connection terminated (4000, ''), reconnecting warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3] Failed to determine the https port for redirect. Application is shutting down... Starting Requestrr - build v2.1.4 - Hotfix 4 warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. Hosting environment: Production Content root path: /root Now listening on: http://[::]:4545 Application started. Press Ctrl+C to shut down. warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot configuration changed: restarting bot warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot has been restarted. warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot configuration changed: restarting bot [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<<RegisterCommands>b__0>d.MoveNext() [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b__0>d.MoveNext() [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b0>d.MoveNext() [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<b0>d.MoveNext() [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<<RegisterCommands>b__0>d.MoveNext() [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b__0>d.MoveNext() [2024-02-11 01:11:36 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b0>d.MoveNext() [2024-02-11 01:11:37 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<b0>d.MoveNext() [2024-02-11 01:11:37 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<<RegisterCommands>b__0>d.MoveNext() [2024-02-11 01:11:37 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b__0>d.MoveNext() [2024-02-11 01:11:39 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b0>d.MoveNext() [2024-02-11 01:11:39 +02:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<b0>d.MoveNext() warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot has been restarted. [2024-02-11 04:19:29 +02:00] [111 /ConnectionCl] [Crit ] Connection terminated (4000, ''), reconnecting

thomst08 commented 7 months ago

Thanks for posting your error log, I have seen an error similar to what you have and I think I have replicated it. I will need to test some more to verify the issue then correct it, but can I ask if you have "Hide slash command request messages in channels" enabled?

andrewmcd7 commented 7 months ago

I believe they are enabled. I just briefly looked and cannot see any settings for that. Where would it be? And should they be enabled or disabled.

thomst08 commented 7 months ago

It can be enabled or disabled, I was wondering if that is the culprit, but from testing I don't think so, it's in the "Chat client" section.

It appears to be with the building of the slash commands when the bot is reset, my guess there could be a network issue and this might be part of the issue as it's not reconnecting to discord... Do you have any special network setting for your containers? And do you have any roles setup in the "Chat clients" section?

andrewmcd7 commented 7 months ago

Let me find out about the special network settings.

No special roles.

I invite the bot to discord, add the channel ID TO REQUESTRR and everything is good for about a day before it starts erroring.

andrewmcd7 commented 7 months ago

We do have I think 5 or 6 requestrrs on 1 host. All named differently. Req1, req2, req3, etc.

I have reached out to the guy who sets them up to see if there’s any weird network settings he’s using.

andrewmcd7 commented 7 months ago

No weird network settings on the containers in portainer.

andrewmcd7 commented 7 months ago

Here’s some more logs on a different container. Same issue:

     at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
     at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
     at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
     at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
     at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
     at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
     at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
     at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
     at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
     at Requestrr.WebApi.RequestrrBot.DownloadClients.Radarr.RadarrClientV3.HttpGetAsync(HttpClient client, RadarrSettings settings, String url) in D:\Users\thoms\Documents\GitHub\requestrr\Requestrr.WebApi\RequestrrBot\DownloadClients\Radarr\RadarrClientV3.cs:line 431
     at Requestrr.WebApi.RequestrrBot.DownloadClients.Radarr.RadarrClientV3.FindExistingMovieByMovieDbIdAsync(Int32 theMovieDbId) in D:\Users\thoms\Documents\GitHub\requestrr\Requestrr.WebApi\RequestrrBot\DownloadClients\Radarr\RadarrClientV3.cs:line 405

fail: Requestrr.WebApi.RequestrrBot.DownloadClients.Radarr.RadarrClient[0] An error occurred finding an existing movie by tmdbId "37541" with Radarr: Name or service not known (radarr:7878) System.Net.Http.HttpRequestException: Name or service not known (radarr:7878) ---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at Requestrr.WebApi.RequestrrBot.DownloadClients.Radarr.RadarrClientV3.HttpGetAsync(HttpClient client, RadarrSettings settings, String url) in D:\Users\thoms\Documents\GitHub\requestrr\Requestrr.WebApi\RequestrrBot\DownloadClients\Radarr\RadarrClientV3.cs:line 431 at Requestrr.WebApi.RequestrrBot.DownloadClients.Radarr.RadarrClientV3.FindExistingMovieByMovieDbIdAsync(Int32 theMovieDbId) in D:\Users\thoms\Documents\GitHub\requestrr\Requestrr.WebApi\RequestrrBot\DownloadClients\Radarr\RadarrClientV3.cs:line 405

thomst08 commented 7 months ago

Hmmm, they are interesting errors, I will do some more testing and try to find a reason for this.

Where the containers working perfectly on version 2.1.2 of requestrr? (about two months back)

andrewmcd7 commented 7 months ago

No. We have been plagued by this for a while.

thomst08 commented 7 months ago

This might take me a while to pin point. I will keep thinking about this and try to come up with an answer.

Just to double check, are all the services you connect requestrr too all on the one server? Is there any known network issues such as drop-outs with the server?

andrewmcd7 commented 7 months ago

Any progress? Not sure what else we can try and do.

thomst08 commented 7 months ago

Hey @andrewmcd7,

Not at the moment, this is still in the back of my mind and I am looking for issues but haven't seen any links to what you have been expecting.

There is a few things to note however, I have a few long term plans, one of which is to setup requestrr to allow to handle multiple sources (multiple radarr connections and using categories to control each one) and to also change how the slash commands are created.

Both of these are going to be a while and might help your setup, but also don't solve the underlying issue your having, at the moment.

I was still wondering if any of the traffic was through a vpn setup on the network that might be part of the issue, but I'm not sure. I'll keep exploring, if I find something I will let you know.

thomst08 commented 7 months ago

Hey @andrewmcd7,

Sorry this has taken me a while, I think I have some solutions to your problem, I don't think this is an issue with the program, but it might be the setup...

First, the second issue you mentioned, this issue has been fixed in the last update, this is because some movies or shows get removed from the TVDB or TMDB and then requestrr can no longer match them, the newest update removes these form the notifications, this should not have been stopping the program, but does fill the logs.

The first issue you noted... I haven't found an EXACT reasons, but this might be to do with your settings or a miss configuration. I managed to emulate this issue by modifying my language file, this didn't give me your exact issue, but something close. If you are using a custom language file, please recreate it from one of the latest version as new properties have been added. If that's not the case, there is a chance your settings might be causing the issue as some in your config will get loaded into the commands, this could cause the issue, it might be worth clearing the settings and rebuilding them, or tweaking each setting and removing weird characters if you have any ()[],\ That could upset the building of the commands.

Let me know if any of this helped, I will keep exploring our options if you are still having issues.

andrewmcd7 commented 7 months ago

Thank you! I'm not sure what you are referencing wrt language file, we haven't modified the config. If it helps, without touching any settings restarting it a few times seems to push past the error. Is it possible this error is some sort of discord IP or bot rate limit?

thomst08 commented 7 months ago

That's ok, in short, you can setup custom language files for you bot, allowing you to setup for another language if needed, however, if you are just using English, then this can be ignored. It is possible this is a rate limit yes, I forgot to mention that sorry, I haven't found a way to trigger this yet, but there is a fix coming. DSharp is releasing a new update soon that fixes a rate limit issue, when it's out, I will update the program, we can see if that helps.

Again, it could be a discord error still, but from the errors, it appears to be a building error of the slash commands or at least when its sent to Discord it is rejected.

edrock200 commented 7 months ago

Just to add to this, seeing similar errors and restarting several times or proxying seems to resolve, which lends itself to the theory that a rate limit is occuring. One thing I've noticed is right at the start of requestrr, in addition to registering the bot, it attempts to push out any backlogged notifications in rapid succession. Some of them are for users that no longer exist. Deleting notifications.json resolved this, but I'm wondering if combination of the two is also causing rate limiting. Would it be possible to delay the notifications and prune notifications that return "user not found?" @thomst08

thomst08 commented 7 months ago

Hey @edrock200, Thanks for your input, that is a good idea, I will look into that part. I'm not sure a delay is the right step, but handling missing users should be something to look into.... I'll definitely dig into this

edrock200 commented 7 months ago

Hey @edrock200, Thanks for your input, that is a good idea, I will look into that part. I'm not sure a delay is the right step, but handling missing users should be something to look into.... I'll definitely dig into this

Thank you! Delay was the wrong word. I meant a throttle of notification alerts, e.g. no more than xx a minute.

thomst08 commented 7 months ago

Hey everyone, just a small update.

Still testing and make a link, I looked over the notifications part, and while that can trigger a lot of requests, I have found that if a user is no longer on your server, it should remove the request once its active, if there is more then 10~ requests it will rate limit, report it in the long and try again in another minute, This part seems to be handling fine and isn't taking down the main bot as it seems to still be functioning. I don't think it in the notification side causing a limit problem in this case by I will keep testing here in other situations to best work it out...

I have, however, started to look over the main resetting of the bot and setup, it seems if there is a break in the slash commands the bot should restart and try to rebuild itself, however, there's a chance this doesn't always happen. So, I am looking into a way of check if the bot broken down and failed to reset and reset itself if this happens, there was no check if DSharp's library fails. I have been keeping an eye on DSharps releases and still waiting for their next release that will have some changes to the rate limiter and a few other things to check if the bot is connected. I might wait until this is released before updating the program, but I will keep testing and try to come up with a solution that works for everyone.

TLDR: Tested notifications, found some cases where there could be an issue, not confident enough in the fix, needs more testing, working on a fix.

edrock200 commented 7 months ago

Thank you!

thomst08 commented 6 months ago

Small update, looking into this a bit more. The main error when registering the slash commands is building the commands, then sending it off to DSharp to send it to Discord, while I can attach error handling functions, none of them seem to catch this error in DSharp, once DSharp errors, this seems to crash DSharps slash register completely, to the point if the bot is reset in program it does not re-register, it appears to not even respond or attempt to register the commands again...

Hopefully when the next update is out this might correct this issue, but DSharp v5 will allow registering dynamic slash commands during run time witch will hopefully solve the root of this issue. (I hope to not have to wait too long to resolve this)

edrock200 commented 6 months ago

Small update, looking into this a bit more. The main error when registering the slash commands is building the commands, then sending it off to DSharp to send it to Discord, while I can attach error handling functions, none of them seem to catch this error in DSharp, once DSharp errors, this seems to crash DSharps slash register completely, to the point if the bot is reset in program it does not re-register, it appears to not even respond or attempt to register the commands again...

Hopefully when the next update is out this might correct this issue, but DSharp v5 will allow registering dynamic slash commands during run time witch will hopefully solve the root of this issue. (I hope to not have to wait too long to resolve this)

Thanks for the info and hard work! In the meantime is their a specific phrase we can grep for in the logs when it found, to trigger a log clear and restart?

thomst08 commented 6 months ago

@edrock200, good question. I would look for "There was an error registering application commands". Once this happens, you loss your slash commands or the bot becomes unresponsive.

thomst08 commented 6 months ago

Hey @andrewmcd7 and @edrock200,

Just giving you both a heads up, I have just released version 2.1.6, this contains a few things that link back to these issues. First, there are a few tweaks in how the bot builds its commands and retrying, these changes may help correct your issues or at lease help prevent others and maybe give us more context.

I have also changed where the temp files are created when building the commands, again, this might help prevent your issue.

I don't expect this version to solve your issues as it does not solve the underlining issue, but it might help, please let me know if there are any changes and post any new logs you have.

edrock200 commented 6 months ago

Hey @andrewmcd7 and @edrock200,

Just giving you both a heads up, I have just released version 2.1.6, this contains a few this that link back to these issues. First, there are a few tweaks in how the bot builds its commands and retrying, this changes may help correct your issues or at lease help prevent others and maybe give us more context.

I have also changed where the temp files are created when building the commands, again, this might help prevent your issue.

I don't expect this version to solve your issues as it does not solve the underlining issue, but it might help, please let me know if there are any changes and post any new logs you have.

Thank you so much!

CapnClunge commented 6 months ago

I've been getting the same error pretty consistently for a few weeks now; only about 1 in every 5 requests actually works. Restarting the container doesn't seem to help, nor does restarting the whole server. It just starts working again seemingly at random. This also occurs whether I request in a channel or through direct message. Below is the resulting log after attempting to request the same movie both by title and by TMDB:

fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Error while searching movies with title the one : Not found: 404
      DSharpPlus.Exceptions.NotFoundException: Not found: 404
         at DSharpPlus.Net.DiscordApiClient.CreateInteractionResponseAsync(UInt64 interaction_id, String interaction_token, InteractionResponseType type, DiscordInteractionResponseBuilder builder)
         at Requestrr.WebApi.RequestrrBot.SlashCommands.RequestGroup.MovieByTitle99999(InteractionContext ctx, String movieTitle)
fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Error while searching movies by TmDb Id 10796 : Not found: 404
      DSharpPlus.Exceptions.NotFoundException: Not found: 404
         at DSharpPlus.Net.DiscordApiClient.CreateInteractionResponseAsync(UInt64 interaction_id, String interaction_token, InteractionResponseType type, DiscordInteractionResponseBuilder builder)
         at Requestrr.WebApi.RequestrrBot.SlashCommands.RequestGroup.ByTmdb99999(InteractionContext ctx, Int64 theMovieDbId)
thomst08 commented 6 months ago

Hey @CapnClunge, Thanks for reaching out, this seems to be a different type of error. From your logs, it appears your server is having issues contacting the TMDB servers. I would like to double check if you have any thing filtering your network? I will take a look and keep an eye on this issue and see if I can work something out.

CapnClunge commented 6 months ago

I was using a pihole and it never showed that it was blocking any requests from the container, but to be extra sure I tried disabling filtering and even switched to a different DNS altogether. I also rebooted the server, modem, and router. Throughout this process I tested for packet loss and found none. Despite all this, I continue to get the "the application did not respond" message the majority of the time. However, if I request the same title repeatedly it will sometimes work. This inconsistency leads me to believe it's not a filtering issue but something else altogether. I don't want to hijack this thread if you think it's an unrelated issue but please let me know if there's anything else you'd like me to try or more information I can provide. Thanks!

thomst08 commented 6 months ago

@CapnClunge, That's more then enough testing, thank you for checking, I think this might be closer to issue #22 I'll see if I can emulate this issue.

edrock200 commented 6 months ago

@thomst08 - since 2.1.6, haven't seen the issue. Thank you so much!

DrunkenMonk0001 commented 4 months ago

From my experience this happens consistently still after an app restart where there's 2 separate discord servers involved - only way I've found to fix it thus far is by removing one of the channel IDs from the "Channel(s) to send notifications to" section, saving then re-adding it again.

thomst08 commented 4 months ago

Hey @DrunkenMonk0001,

Thanks for the input, that makes sense, removing a channel then re-adding it would cause the bot to restart due to a setting change, so for some reason there is a disconnect between the bot and Discord.

Just want to double check, is this one bot to two different Discord servers? Do you mind posting a copy of your log when this occurs before you fix the issue?

DrunkenMonk0001 commented 4 months ago

One bot to two different discord servers, yup. Unfortunately though, the console doesn't output any log when the interaction occurs at all. I need to do the removal and addition of the channel ID to have it begin displaying anything, by which point it begins to work normally. Here's the log anyway;

Starting Requestrr - build 'v2.1.6'
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
Hosting environment: Production
Content root path: **
Now listening on: http://[::]:4545
Application started. Press Ctrl+C to shut down.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
      Failed to determine the https port for redirect.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.

*edit:

then here's the log after I remove one of the channel IDs and re-add, the fail happens after the ID is removed*

warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Error settings up bot
Index was outside the bounds of the array.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
[2024-05-28 12:55:33 +01:00] [105 /WebSocketRec] [Warn ] Unknown event: GUILD_AUDIT_LOG_ENTRY_CREATE
thomst08 commented 4 months ago

Thanks @DrunkenMonk0001, that gives me abit more to work with. :)

Znuff commented 3 months ago

I've had this happen even on old versions. Still happening to this day.

I've had it happen randomly when I just restarted the container. I've had it happen randomly when it lost internet connection.

Restarting the bot numerous times did nothing.

I had a ticket on the old repository: https://github.com/darkalfx/requestrr/issues/185

Sometimes I can fix it by changing the language to something else (say Chinese), and then back to English. But it's a hit & miss, so not really a proper fix.

edrock200 commented 3 months ago

Another potential cause, I had added one bot to several friend's cords so they could request stuff. It seems if you add it to more than a few, it can cause this error, especially if you have a lot of categories. I kicked it from a few cords and was fine again. So in the rare case this is your scenario, best to make a requester instance and bot for each cord.

Znuff commented 3 months ago

I have it on servers with a big user count, I was under the impression that might also count, but yes, multiple servers might trigger this issue.

thomst08 commented 3 months ago

These are good points, from my testing, I've found it seems to be a fail in D# when connecting to Discord. The issue is I could find a way of knowing when an error occurred and catching it to restart the setup process again. What your describing here does add up.

Currently the bot will watch the settings for any changes and then resets all the commands, it also looks out for new servers and ones dropping off, this will also trigger a reset. Commands are also build dynamically into a temp file and used to connect to each server, if the command file fails to build, it will restart the process. There's a delay in-between the checks of about 60 seconds, but if there is a number of servers and maybe a number of commands, this could be hitting the rate limit in D# and Discord, that, or Discord fails receiving the commands.

I was waiting for D# v5 to come out (and for the documentation to be updated, looks like there is a migration guide now) as it seems to have a few new things I can use and some changes to the rate limits and dynamic command building. What I might do is turn my attention back to this issue and see if I can implement the nightly builds into a version of Requestrr, I could put it into a docker container for anyone to test and see if that helps resolve the issue.

Would anyone be OK to test such a build?

edrock200 commented 3 months ago

These are good points, from my testing, I've found it seems to be a fail in D# when connecting to Discord. The issue is I could find a way of knowing when an error occurred and catching it to restart the setup process again. What your describing here does add up.

Currently the bot will watch the settings for any changes and then resets all the commands, it also looks out for new servers and ones dropping off, this will also trigger a reset. Commands are also build dynamically into a temp file and used to connect to each server, if the command file fails to build, it will restart the process. There's a delay in-between the checks of about 60 seconds, but if there is a number of servers and maybe a number of commands, this could be hitting the rate limit in D# and Discord, that, or Discord fails receiving the commands.

I was waiting for D# v5 to come out (and for the documentation to be updated, looks like there is a migration guide now) as it seems to have a few new things I can use and some changes to the rate limits and dynamic command building. What I might do is turn my attention back to this issue and see if I can implement the nightly builds into a version of Requestrr, I could put it into a docker container for anyone to test and see if that helps resolve the issue.

Would anyone be OK to test such a build?

I'd be happy to!

thomst08 commented 2 months ago

Hey guys, I've looked over DSharps latest nightly build and have got it running, however, it is bit unstable at the moment... Basically, the bot cannot restart on a setting change and fails completely, the only way forward with that is to ether change the handling of a restart and/or wait for a more later build. As it's still in development I will leave it alone for the time being and wait for a full release or a newer build to dev on, as I will need to rebuild the slash commands in their new library (that will require a bit more work). Testing however did give me some insight on some other areas, I still have yet to complete the Lidarr integration and its nearing its completion.

My intent is to come back to this bug and find a solution, I'm sorry this issue is taking a while. I will comment here when a build is ready to test, if not fix the problem.

andrewmcd7 commented 2 weeks ago

Has there been any progress on this? I’m getting it failing super often now.

[2024-09-16 14:10:22 +03:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<<RegisterCommands>b__0>d.MoveNext() [2024-09-16 14:10:22 +03:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b__0>d.MoveNext() [2024-09-16 14:10:22 +03:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b0>d.MoveNext() [2024-09-16 14:10:22 +03:00] [0 / ] [Crit ] There was an error registering application commands System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<b0>d.MoveNext() warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot has been restarted.

thomst08 commented 2 weeks ago

Hey @andrewmcd7,

Partly, the latest version has an updated version of Dsharp+, I haven't looked much more into this issue as I wanted to complete lidarr support first, but I will begin researching this issue again.

Has this happened since updating to the new version?

andrewmcd7 commented 2 weeks ago

Yes it’s happening daily. Sent from my iPhoneOn Sep 16, 2024, at 7:18 AM, thomst08 @.***> wrote: Hey @andrewmcd7, Partly, the latest version has an updated version of Dsharp+, I haven't looked much more into this issue as I wanted to complete lidarr support first, but I will begin researching this issue again. Has this happened since updating to the new version?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

andrewmcd7 commented 2 weeks ago

This bot currently will not submit my request to the arr’s.. I have redeployed in docker numerous times.  And rebooted it numberous times. I just cannot get it to respond.warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot configuration changed: restarting bot[2024-09-16 14:43:55 +03:00] [0 / ] [Crit ] There was an error registering application commandsSystem.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<<RegisterCommands>b__0>d.MoveNext()[2024-09-16 14:43:55 +03:00] [0 / ] [Crit ] There was an error registering application commandsSystem.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b__0>d.MoveNext()[2024-09-16 14:43:55 +03:00] [0 / ] [Crit ] There was an error registering application commandsSystem.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>cDisplayClass39_0.<b0>d.MoveNext()[2024-09-16 14:43:55 +03:00] [0 / ] [Crit ] There was an error registering application commandsSystem.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray') at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) at System.Collections.Generic.List1.set_Capacity(Int32 value) at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection) at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<b0>d.MoveNext()warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3] Failed to determine the https port for redirect.warn: Requestrr.WebApi.RequestrrBot.ChatBot[0] Bot has been restarted.Sent from my iPadOn Sep 16, 2024, at 7:18 AM, thomst08 @.***> wrote: Hey @andrewmcd7, Partly, the latest version has an updated version of Dsharp+, I haven't looked much more into this issue as I wanted to complete lidarr support first, but I will begin researching this issue again. Has this happened since updating to the new version?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

thomst08 commented 2 weeks ago

@andrewmcd7,

I think I have something! I did a bit more digging and found some errors matching yours, it seems when you have a number of servers with commands, DSharp+ has issues dealing with this kind of setup if you assign the same commands to all the guilds, the answer is to set them up as global commands.

So.... knowing this now and having a look, in its current form, Requestrr assigns the same commands to all servers with each of their guild numbers.... So I quickly setup a version that only assigns the commands as global commands, the draw back is you can only have 100 of them. So as long as you don't have over 98~ categories, you should be ok....

I have setup a docker container for you to try this new version. thomst08/requestrr:dev Give this container a try and let me know if that solves the issue then we can go from there. Anyone else who is having this issue is also welcome to try too.

kevle1 commented 2 weeks ago

I was also getting the following exception, and yep I did have the bot in multiple servers -

[Crit ] There was an error registering application commands
System.ArgumentException: Source array was not long enough. Check the source index, length, and the array's lower bounds. (Parameter 'sourceArray')
   at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable)
   at System.Collections.Generic.List`1.set_Capacity(Int32 value)
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass39_0.<<RegisterCommands>b__0>d.MoveNext()

Trying out the dev tag container it looks good, commands are registered again. Cheers!

edrock200 commented 2 weeks ago

@andrewmcd7,

I think I have something! I did a bit more digging and found some errors matching yours, it seems when you have a number of servers with commands, DSharp+ has issues dealing with this kind of setup if you assign the same commands to all the guilds, the answer is to set them up as global commands.

So.... knowing this now and having a look, in its current form, Requestrr assigns the same commands to all servers with each of their guild numbers.... So I quickly setup a version that only assigns the commands as global commands, the draw back is you can only have 100 of them. So as long as you don't have over 98~ categories, you should be ok....

I have setup a docker container for you to try this new version. thomst08/requestrr:dev Give this container a try and let me know if that solves the issue then we can go from there. Anyone else who is having this issue is also welcome to try too.

You sir, are a genius! Thank you!