search

marsara9 / lemmy-search

An enhanced search engine just for Lemmy/Fediverse
https://www.search-lemmy.com
GNU Affero General Public License v3.0
81 stars 4 forks source link

Add public API to expose bots. #46

Open marsara9 opened 1 year ago

marsara9 commented 1 year ago

Idea: beyond just search results, expose APIs that do some analysis on the data that's already collected. Such as returning a list of bot accounts that auto-post articles.

rcmaehl commented 1 year ago

Hopefully, most bots will be on the easy-to-use lemmybots.com (and other bot specific instances) soon.

marsara9 commented 1 year ago

Hopefully. But I'm aiming to try and do some data analysis to be able to expose a public API that can then be used to keep an up-to-date listing of "bad bots". This isn't going to be an easy problem to solve however. As under NO CIRCUMSTANCE can I have any false positives here. Mainly I cannot list a real user in the response. I'd rather let a bot slip through.

My end goal here by the way, is now that I have a decent index of the Fediverse, to try and see what sort of analysis I can do on that data to provide value back to the community. Be it bot identification, or anything else.

For example, another endpoint I can potentially expose is a list of NSFW communities. As right now, Lemmy doesn't support a way to flag an entire community as NSFW, but it would be trivial to return a list of communities where 90+% of the content is NSFW. This could then be fed into a script that's run user-side that automatically blocks (or subscribes) to said communities.