ShokoAnime / ShokoServer

Repository for Shoko Server.
http://shokoanime.com/shoko-server/
MIT License
374 stars 74 forks source link

Implement notify system from AniDB UDP API #1129

Open Terrails opened 1 month ago

Terrails commented 1 month ago

Implements the AniDB notify system which will be useful to read and handle file moved messages.

There are 2 new tables:

Currently there is no need for something like a AniDB_Notification table as there is no real use for it. Unread notifications are only added to the AniDB_NotifyQueue table and there is nothing that fetches further notification content at the moment.

And 3 new config options:

GetAniDBNotifyJob should run in the configured intervals or when the new /Action/GetAniDBNotifications/ v3 API endpoint is ran. This PR just lays down the groundwork for fetching and saving unread message content to the database. There is already a IsFileMoved flag in AniDB_Message that will be used for future behavior. It includes the logic to refetch AniDB data for a moved file now.

revam commented 1 month ago

Also, before I forget. We might need a guard to prevent too often requests for the endpoint, since iirc the endpoint have a rate limit of 20 minutes per request, and we want shoko to adhered to that rule.

Edit: Wiki says 20.

revam commented 1 month ago

@Terrails can you add the job to act on the File Moved flag to this PR? And also an option to auto schedule it if a setting is set, and an endpoint to schedule all files that have been moved but not acted on yet from an action endpoint?

Terrails commented 1 month ago

Also, before I forget. We might need a guard to prevent too often requests for the endpoint, since iirc the endpoint have a rate limit of 20 minutes per request, and we want shoko to adhered to that rule.

Edit: Wiki says 20.

Shouldn't that already be somewhat handled by https://github.com/ShokoAnime/ShokoServer/blob/906219e999809c1e138b792c3b1085d6355928c0/Shoko.Server/Services/ActionService.cs#L940-L962 It does stop it from running unless it was forced, which happens only when the API endpoint is manually executed. The automatic scheduling should not run more often than configured as the forced refresh flag is false. https://github.com/ShokoAnime/ShokoServer/blob/906219e999809c1e138b792c3b1085d6355928c0/Shoko.Server/Server/ShokoServer.cs#L343

Terrails commented 1 month ago

@Terrails can you add the job to act on the File Moved flag to this PR? And also an option to auto schedule it if a setting is set, and an endpoint to schedule all files that have been moved but not acted on yet from an action endpoint?

Should be implemented now. I assume that ProcessFileJob has to be executed to fetch the new file data https://github.com/ShokoAnime/ShokoServer/blob/aacf117c7825a0fb39a97e26fb8b4c45199f7ed0/Shoko.Server/Scheduling/Jobs/Shoko/ProcessFileMovedMessageJob.cs#L43-L56 And then if it is successful it is flagged as handled. The job also has a new endpoint now and should automatically run after new notifications are fetched from AniDB https://github.com/ShokoAnime/ShokoServer/blob/aacf117c7825a0fb39a97e26fb8b4c45199f7ed0/Shoko.Server/Services/ActionService.cs#L965-L967 It might be a good idea to add a force flag to the API endpoint here though: Did that https://github.com/ShokoAnime/ShokoServer/blob/aacf117c7825a0fb39a97e26fb8b4c45199f7ed0/Shoko.Server/Services/ActionService.cs#L973