Missing kbin communities?

[csm10495]

Hey folks,

I searched for "squaredcircle" and got:

This seems to be missing the SquaredCircle community on kbin: https://sh.itjust.works/c/SquaredCircle@kbin.social.

Any idea why it didn't pop up? I have 'selected all' instances. Do kbin communities not show up?

[tgxn]

Cause I only currently scan instances from lemmy instances :)

I'll take this as a feature request to scan KBin magazines too - but this will require mre effort (since I need to write an implementation that gets the same details as Lemmy returns) and then format it for display on the frontend.

Probably a couple days, but no promises. 🤞

[tgxn]

Notes on what I'm thinking:

• KBin magazines will be hidden by default, and I'll have a menu option to display them.

Also, quick checking none of the KBin APIs work ? I'm getting errors from everything, maybe something I'm doing wrong

I'm looking at the docs here https://docs.kbin.pub/#get-single-magazine

kbin.social/m/SquaredCircle

[tgxn]

OK this works

[csm10495]

Ayee there ya go. I would recommend going the other direction and having kbin ones be included by defaults and opt-out able.

At the end of the day: fediverse is fediverse and cross-pollination between lemmy and kbin should work out just fine. My other qualm is that I personally have missed out on communities from barely knowing what kbin was. I don't think regular users should care or know the difference.

(At the end of the day, your choice though) :)

[tgxn]

Still an issue, that the endpoints to returns lists of magazines don't seem to work :/ Which would stop me crawling for any communities.

At the end of the day: fediverse is fediverse and cross-pollination between lemmy and kbin should work out just fine. My other qualm is that I personally have missed out on communities from barely knowing what kbin was. I don't think regular users should care or know the difference.

Yeah I don't really disagree, I was more trying to make it simpler for new users; I think as long as I can differentiate the view (like the card could be purple or something and clearly say it's a KBin community) then that should be ok to include by default.

[tgxn]

Just running into roadblocks with kbin's API 😠

[csm10495]

Yeah.. i'm really confused as to how federation works. Like I thought somehow Lemmy/Kbin would have similar APIs for communities, but yeah, I guess not. I can't seem to find any way to get their magazine list api to work either.

[tgxn]

Super frustrating, looks like they have some work on new APIs here: https://codeberg.org/Kbin/kbin-core/pulls/357

Essentially, I don't hit the "Federation" endpoints, I just need a paginated listing of the communities that are on each instance.

Looks like the workaround with requesting application/ld+json doesn't work on /magazines :/

EDIT: More stuff here https://github.com/tgxn/lemmy-explorer/pull/101#issuecomment-1617383548

[csm10495]

Here is a 'terrible' way to get all the magazines on a given kbin instance (via a good-ol bash script):

#!/bin/bash
# Parses all kbin magazines listed on a given instance
# Pass the results to 'grep -v @' to filter out magazines on other instances

INSTANCE=$1

if [ -z $INSTANCE ]; then
    echo "Provide an instance as the only parameter (like kbin.social)"
    exit 1
fi

function parse() {
    PAGE=$1
    curl --fail -s https://${INSTANCE}/magazines?p=$PAGE | grep form | grep '/subscribe' | sed 's/.*="\/m\/\(.*\)\/subscribe"/\1/g'
    return ${PIPESTATUS[0]}
}

for idx in $(seq 10000); do
    if ! parse $idx; then
        break
    fi
done

Basically it walks the magazines list html and parses for /subscribe then pulls out the magazine name. It takes about a minute or so.. but I'm not sure its a great way to handle this.

Here's a current list it pulled (sort/uniq'd) via:

./scrape_kbin_communities.sh kbin.social | sort | uniq > kbin.social_communities.txt

kbin.social_communities.txt

Total of 9058 magazines found. 4363 local ones.

[tgxn]

Getting somewhere now! :D - Sketchy method I am not a huge fan of, I implemented it directly lmao

It's nice your script returns non-local communities too (kbin.sh will return gaming@kbin.social) but I am going to filter those and then create follow-up jobs to scan them.

[tgxn]

just deploying it now... 🤞

[csm10495]

Looks good! Thanks! Hopefully a real API works eventually lol.