search

CuddleBear92 / Hydrus-Presets-and-Scripts

collection of presets and scripts for Hydrus
267 stars 53 forks source link

fix(downloaders): replace outdated baraag downloader #171

Open blackphillip-hkp opened 1 month ago

blackphillip-hkp commented 1 month ago

Replaces the current broken baraag downloader with one that works as of 2024-10-03.

floogulinc commented 1 month ago

Does this use the mastodon API?

blackphillip-hkp commented 1 month ago

@floogulinc Yep, with a similar lookup strategy as the Fanbox downloader using the Fanbox API (actually used it and the ExH downloaders as inspiration). I tested this downloader with over 100 queries downloading over 50,000 status attachments over the last 24 hours without any errors or any of baraag's infamous rate limits except a handful of unresolvable "media proxy" attachment links.

I also skipped over the included "mastodon parsers" and built custom parsers just for baraag, but I'm pretty sure these artist id lookup and status attachments parsers could be applied to any mastodon site.

Seems like a downside of connecting the mastodon API and Hydrus is there will always be one ignored request. I'm new to building downloaders, and while it seems pretty straightforward maybe there's a way to avoid that extra request that I'm not thinking of.

floogulinc commented 1 month ago

I've had a generalized mastodon/Pleroma API parser in the discord for a while now which includes baraag. I just haven't added it here yet because I was waiting on a feature that was only added to hydrus in the latest version to be able to properly handle getting notes of the post content without just dumping the html into the note. You might want to check that out too, a lot of parser work just happens in the discord now rather than here.

blackphillip-hkp commented 1 month ago

Neat, I'm not a big discord user but I'll check it out :) I'm currently tweaking the EH/ExH downloader, I've been having some issues with it too, but this might be due to an accidental double-import on my part. It reports something like Cannot parse zzz - renamed due to auto-import - Could not find a parser for zzz - renamed due to auto-import - so-and-so URL Class!