casual-simulation / casualos

Casual Open Simulation for the Web
https://ab1.bot
MIT License
48 stars 9 forks source link

Add the ability to scan files uploaded to S3 for content moderation #520

Closed KallynGowdy closed 1 month ago

KallynGowdy commented 1 month ago

Currently, there is no content moderation supported for uploaded files. Ideally, we would be able to use AWS Rekognition to perform content moderation on uploaded files.

It should be possible to configure the server to automatically perform content moderation on files uploaded to S3 and specify labels and confidence values (per label) that should be able to trigger notification and/or removal of the offending object.

Additionally, a list of banned hashes should be stored in the database that is searched whenever a new item is attempted to be uploaded.