mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.37k stars 925 forks source link

What url to use when downloading profiles of bluesky #5246

Closed Nicosow2 closed 5 months ago

Nicosow2 commented 6 months ago

i tried but it did the error of unsupported url

mikf commented 6 months ago

https://bsky.app/profile/HANdLE for example https://bsky.app/profile/bsky.app

Make sure you are using the latest release (v1.26.8) Run gallery-dl --version to check.

Technotron21 commented 6 months ago

On something of a related note: how exactly would I be able to batch download every user on the following page like I can on twitter? Every attempt I've made doesn't even even prompt any sort of response, unlike every other error I've seen.

Also, is it possible to download reposts from an account? Given the lack of any presence for the site on the default config currently, I wouldn't be surprised if it's too new to implement yet.

mikf commented 6 months ago

On something of a related note: how exactly would I be able to batch download every user on the following page like I can on twitter? Every attempt I've made doesn't even even prompt any sort of response, unlike every other error I've seen.

That's a bug. Or rather the result of me forgetting to add a line of code. As a workaround, you can set blacklist to an empty string (-o blacklist=) or you write all its URLs to a file and then use that as input.

gallery-dl -o blacklist= https://bsky.app/profile/bsky.app/follows
gallery-dl -g https://bsky.app/profile/bsky.app/follows > urls
gallery-dl -i urls

Also, is it possible to download reposts from an account? Given the lack of any presence for the site on the default config currently, I wouldn't be surprised if it's too new to implement yet.

Use the /posts feed/timeline or adjust bluesky.include

The Posts, Replies, Media, etc tabs on bluesky profiles don't have real URLs, but you can select them in gallery-dl by adding the lowercase name after a profile URL (https://bsky.app/profile/bsky.app/posts).

Just https://bsky.app/profile/HANDLE alone uses the Media feed by default.

gallery-dl https://bsky.app/profile/bsky.app/posts
gallery-dl -o include=posts https://bsky.app/profile/bsky.app
Technotron21 commented 6 months ago

Those definitely helped, but I am noticing the reposts are being separated by the users that did the original posts, so I wonder if it's possible to have them all under the same folder like with Twitter.

mikf commented 6 months ago

Enable user metadata and use the name from that for directories.

edit: https://github.com/mikf/gallery-dl/commit/311a21bfb2a6cf9c29d1747fa1cfac94ddcedc64

Technotron21 commented 6 months ago

How do I go about doing that? I'm only really used to the more basic commands.

mikf commented 6 months ago

Add this to your config file

        "bluesky": {
            "metadata": "user",
            "directory": ["{category}", "{user[handle]}"]
        }

If you don't have one, run gallery-dl --config-create and copy-paste this into the extractor block.


If you want reposts in their own subdirectory, you could do something like

            "directory": {
                "author['did'] != user['did']": ["{category}", "{user[handle]}", "Reposts", "{author[handle]}"],
                ""                            : ["{category}", "{user[handle]}"]
            }

edit: Reposts are disabled by default in the next release, by the way (https://github.com/mikf/gallery-dl/commit/495c9ee126bd31bdd881a5770232fb2f71384a5e) Add "reposts": true to your bluesky config to keep them enabled.

literallysomeone commented 6 months ago

Hello. I'm wondering how is it possible to download likes from a bluesky account using gallery-dl. It seems to be a supported feature since it's stated in the Supported Sites page, but I can't figure out how to properly tell the application that's what I want to do.

The Posts, Replies, Media, etc tabs on bluesky profiles don't have real URLs, but you can select them in gallery-dl by adding the lowercase name after a profile URL (https://bsky.app/profile/bsky.app/posts).

I tried putting in something along the lines of https://bsky.app/profile/bsky.app/likes barring me using my own account, but I allows get back a [bluesky][error] API request failed (400 Bad Request) spit out at me. Heck, even putting in that example URL also spits out that error. I don't think this is necessarily a problem with Bluesky since like viewing websites still work from what I've seen (in terms of both my personal account and the example URL showing likes).

Am I being dumb and I'm just putting in the wrong URL or is this potentially an actual issue with the app?

mikf commented 6 months ago

These URLs are the right ones, it's just that likes are kind of broken at the moment. With the current method, you can only access your own liked posts, and only when logged in. Furthermore, the feed has no proper end, meaning gallery-dl will request more liked posts indefinitely until you kill the process or Ctrl+C it.

literallysomeone commented 6 months ago

With the current method, you can only access your own liked posts, and only when logged in.

That's strange since I am indeed signed into the account I'm attempting to download likes from where it's pulling cookies from (Firefox in my case), so I'm not too sure why it's still erroring for me in that case. I also just tried exporting my cookies to a text file and setting them to be used instead of the cookies from my browser, but I still get [bluesky][error] API request failed (400 Bad Request). I'm not sure if you mean it technically works barring the issues of needing to be signed in and having to kill the process once done, or it's just bugged and not working at all, but on my end I can't get it to function.

I suppose I'll just wait for a proper fix for this. My main usage of this software is just scraping the likes and shares from across my online social medias, so I'm hopeful this will become given how more and more artists are moving to Bluesky these days.

mikf commented 6 months ago

cookies

https://github.com/mikf/gallery-dl/issues/4438#issuecomment-1937738238

edit: cookies don't work on bluesky. The site itself doesn't use cookies. You need to provide username and password to login, but you can remove password after the first login.

literallysomeone commented 6 months ago

edit: cookies don't work on bluesky. The site itself doesn't use cookies. You need to provide username and password to login, but you can remove password after the first login.

Oh, my mistake. Well now I got it all working now! Apologies for all of my comments here about this today.