search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
10.76k stars 885 forks source link

seiso.party support #1635

Closed thatfuckingbird closed 3 years ago

thatfuckingbird commented 3 years ago

A kemono.party fork/rewrite(?). Not sure how much of the kemono code could be reused.

Some info from their admin:

Main difference isn't in the UI, but in the code that runs the backend and the importer. It was almost completely rewritten (90%+) and the importer is much more reliable when it comes to embedded things and weird formats. This leads to higher quality content on the site.

Additionally, the storage that Seiso uses is a lot different from Kemono's storage which means that it won't buckle under load even when it gets to the amount of traffic that Kemono has. Images should always load regardless of traffic, even large uncached ones.

That's the gist of it.

mikf commented 3 years ago

The site doesn't appear to have a convenient API like kemono.party does, so not much of the current code can be reused, I think. Maybe some from the initial kemono commit that manually parsed HTML, but I doubt it.

Do you know where to find the site's code or any form of documentation for an eventual API?

thatfuckingbird commented 3 years ago

https://paywall.party/seiso/catalog.html is all the info I've found. There is a post from the admin that it might be made open source later but right now it is not. No mention of API, looks like we are out of luck for that.

Looking at the source, parsing the HTML of artist galleries shouldn't be too bad. The individual post pages aren't too bad either, looks like all the files we want have URLs beginning with cdn.seiso.party/files/.

Other than those, extracting the post title and text would be nice, especially that the post html can contain relevant links (e.g. to google drive or other file hosters).

mikf commented 3 years ago

Initial support got added in https://github.com/mikf/gallery-dl/commit/f74cf52e2b88af8932ee90c330b1098596214c7a. It behaves more or less just like the kemono.party extractors as in:

It also always provides username information without enabling a metadata option. This should probably be used instead of the user ID from user, since that doesn't reflect the real ID like it does on kemono.

thatfuckingbird commented 3 years ago

Thank you, appreciate your work a lot! Now I can scratch this off my TODO list.

mikf commented 3 years ago

Quick update: