search

the-draupnir-project / Draupnir

A Matrix moderation bot
https://the-draupnir-project.github.io/draupnir-documentation/
78 stars 14 forks source link

Consider creating an `NSFWImageProtection` that uses `nsfwjs`. #516

Open Gnuxie opened 3 weeks ago

Gnuxie commented 3 weeks ago

The protection would mean we'd have to add https://github.com/infinitered/nsfwjs to our dependencies, that would also download or bundle this model https://github.com/gantman/nsfw_model.

At the moment we will redact the original message and put a notice in the management room about what happened and what the media id/server combo was. We will not provide a clickable link. We will not be able to provide overide support or anything like that at this phase. So if there was a false positive, it'll be tough luck for now until we can get https://github.com/the-draupnir-project/Draupnir/issues/466 worked out.

FSG-Cat commented 3 weeks ago

A PL based exemption logic could work in place of #466 for now is a idea.

Since elevated PL usually means trusted a bit more than those at default PL.

Gnuxie commented 2 days ago

See mjolnir's attempt https://github.com/matrix-org/mjolnir/pull/520

I didn't realize how large tensorflow would be as a dependency, which is annoying because the nsfwjs github seems to scream about how the model is so small, rip i guess.