Dorge47 / DiscordBot

MIT License
1 stars 0 forks source link

Move stream cache to SQL database #4

Open Dorge47 opened 9 months ago

Dorge47 commented 9 months ago

A SQL database seems like a much cleaner and more permanent way to cache data for videos/streams as well as channel metadata. Response objects from the YouTube API will be parsed and sent to the database using the nodejs driver for whatever database technology is used. A secondary system (another rpi) should be used to serve the database so that the performance of the main program is unaffected. It is currently unclear whether a relational or object-store database model would be a better fit. The structure of YouTube's API response is unlikely to change but not impossible. In the case of the channels themselves, there may also be a use case for storing information not found in the API response (such as birthdays or links to profiles on other platforms).

Dorge47 commented 4 months ago

The chosen database language will be MariaDB. The performance impact is negligible so it will be run on the same rpi as the bot. While responses from YouTube and Holodex are incredibly unlikely to change, the format of the database may change as decisions are made about what information they should contain.

Dorge47 commented 3 months ago

Nodejs driver was added in f63fa72, most of the other commits that day were the result of my misunderstanding of how the promise API worked and running into problems returning a promise of a promise. A decision does not need to be made about how often to sync the local cache with the database since ACID-compliant databases update in real time.