Investitage Patreon's download limit and how the extension triggers it

dogpixels / patreon-helper

Downloads media as you browse Patreon, so you don't have to do this manually.

MIT License

22 stars 5 forks source link

Investitage Patreon's download limit and how the extension triggers it #5

Open dogpixels opened 3 years ago

dogpixels commented 3 years ago

User feedback reports that the extension occasionally triggers some download limit on Patreon and prevents the user from downloading media for a certain while.

This limit is to be investigated and the extension's own throttling adjusted accordingly.

Help Wanted

Hints and documentation regarding that download limit on Patreon are highly appreciated.

canadianfree commented 1 year ago

I encountered a bunch of failed downloads today, but I believe the reason is that I had set the download interval so high, and had so many downloads queued up, that it took a long time to get to them and a time-based token in the links has expired. Maybe this is what others encountered.

Relatedly, I don't seem to have a way to clear the queue now of the failed links without reinstalling and losing all history.

dogpixels commented 1 year ago

Thanks for the report. You are right, there is an access token attached to each link, and that expires at some point. I guess you could run into this issue with relatively large downloads; something I lack experience with myself, since the artist I had pledged for usually shared simple images. Knowing that there are larger files hosted on Patreon is a valuable knowledge though, that can be taken into account in a re-implementation of such an addon.

canadianfree commented 1 year ago

The issue wasn't a single large download. What I encountered I think is that I had a large to-be-downloaded queue. I was trying to download too many posts at once. So by the time the addon worked through a lot of links on the queue it encountered some that had been on the queue for several hours and the tokens in the links had expired. At least that's what I believe happened.

canadianfree commented 8 months ago

So, once I've gotten into the situation where I have maybe 500 links queued up that are all expired, do you have any suggestions on how to get out of the situation? None of the options I can think of are ideal:

Turn off addon: when I turn it back on it resumes the expired link downloads.
Disconnect internet and let it run through all the links: takes hours or days. I believe this will also mark all these links as downloaded inside the extension, preventing me from redownloading them.
Uninstall & reinstall addon: halts downloads & clears queue, but loses the history of everything I've successfully downloaded.

I'm wondering if there's a way to manually clear the queue in javascript from the dev tools.

canadianfree commented 8 months ago

I tried to remove the pending linksby opening the addon debugger and running the following line in download.js right after line 36:

request.onsuccess = (event) => { event.target.result.delete(); event.target.result.continue(); }

It looks like line 36 grabs the items not downloaded, so this should remove all pending links.

I ran it several times but it seems to keep trying and failing to download links. I thought it might be new links, but they are failing as expired.