astro / buzzrelay

Source to relay.fedi.buzz: relay the streaming API of Mastodon instances
https://relay.fedi.buzz
GNU Affero General Public License v3.0
70 stars 9 forks source link

Feature: support for language filtering #1

Closed CustardFlan closed 7 months ago

CustardFlan commented 1 year ago

Hi! First, thank you for creating such amazing piece of software!

And, here comes the request

Like the tag or instances following, can you add the option for follow only posts in Mastodon.supported, selected langs, please? like ES,US,JP etc...

It can be very handy for lang-specific instances!

Best regards, Custard

jippi commented 8 months ago

👍 to this one - being able to to subscribe to https://fedi.buzz/api/v1/streaming/public via posting language would be huge.

Ideally, something along the lines of https://relay.fedi.buzz/language/en (english), https://relay.fedi.buzz/language/da (danish) and so forth.

As a small instance, which is primarily focused on Scandinavia, being able to subscribe to English, Danish, Norwegian, and Swedish from the "firehose" would be an amazing capability and improve the reach and user content experience quite significantly.

Today, I have to subscribe to big instances via https://relay.fedi.buzz/instance/, that firehoses a huge amount of content that my users will never see/care about, only for it to be ingested, stored, and eventually pruned out again a couple of weeks later as the posts expire. Doing that, has some pretty significant cost implications as well, that would be great to avoid :)

And while its likely going to be imperfect, since it relies on the post language provided by users, it would still be a meaningful quality of life and cost improvement over status quo.

Thank @astro

astro commented 7 months ago

So far I have understood this feature request to add language filtering to the existing endpoints. Would you like endpoints with all posts for a specific language instead? That would be a bit easier to add.

jippi commented 7 months ago

I would be perfectly happy with a new endpoint emitting all posts with a specific language :)

The combination of tag+language would be A++, but also fully recognise its a larger shift and lift than a new endpoint dedicated only to language.

In my specific use-case, I would like to firehose all danish posts from the relay firehose into my danish server, so its easier to discover trends and improve coverage of language specific conversations - regardless of any tags - just purely on the language identifier

astro commented 7 months ago

Give this relay endpoint a try:

https://relay.fedi.buzz/language/da

To retain scale and growth at maintainable scope for a side-project, please don't follow the following languages as their throughput is immense:

Use our public live streaming API if you want to drink from the Firehose.

jippi commented 7 months ago

Thanks @astro ! Sorry for the delayed response here, I really do appreciate you implementing this 🎉