search

SuperGouge / ChanThreadWatch

Fork of the original discontinued ChanThreadWatch.
90 stars 13 forks source link

Add a site helper that follows the meta refresh #78

Closed DerSandmann-Badcode closed 6 years ago

DerSandmann-Badcode commented 6 years ago

Maybe we should pull this into a base feature if more sites use this

image

SuperGouge commented 6 years ago

The thing that bothers me is that this makes an HTTP request inside the GetImages function that is not supposed to require connectivity and just works off the HTMLParser. I think finding a way to gracefully detect this metadata element and treat it like an HTTP redirect inside the downloading functions would be better. It would also work for other sites as well, as you suggested.

DerSandmann-Badcode commented 6 years ago

Yep let me throw that together.

It was just a bit troublesome because the downloader tries to immediately save the result to disk (Instead of images, you get HTML saved as an image). We'll have to parse the result and then actually look at the content type that comes back, then do a second request if there's a refresh.

On 9 Mar. 2018 2:05 pm, "SuperGouge" notifications@github.com wrote:

The thing that bothers me is that this makes an HTTP request inside the GetImages function that is not supposed to require connectivity and just works off the HTMLParser. I think finding a way to gracefully detect this metadata element and treat it like an HTTP redirect inside the downloading functions would be better. It would also work for other sites as well, as you suggested.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/SuperGouge/ChanThreadWatch/pull/78#issuecomment-371678661, or mute the thread https://github.com/notifications/unsubscribe-auth/AH1oovSvr7G_GszK3fmtVRMcrTRhHNayks5tcdVAgaJpZM4SeipT .