search

dteviot / WebToEpub

A simple Chrome (and Firefox) Extension that converts Web Novels (and other web pages) into an EPUB.
Other
679 stars 132 forks source link

I don't know what happened to this site https://novelsemperor.net #1471

Open sksmsalman opened 1 week ago

sksmsalman commented 1 week ago

Describe the bug I don't know if it's bug or not when I use the extension it shows something like this

It shows the following message

WARNING: Site 'novelsemperor.net' has sent an Access

Denied (403) error. You may need to logon to site, or browse site normally until you get a Cloudflare "Are you a human" page or satisfy some other CAPTCHA before WebToEpub can continue. Open Page?

Expected behavior To work normally

Additional context

It was working just fine the day before yesterday don't know if something has changed tried different 0.0.173 test version but still the same also used vpn and also changed to different browser but the same

Same happened to fanmtl.com but using vpn it was avoided but snice some time the speed of crawling has been slowed down to extreme

gamebeaker commented 1 week ago

Same happened to fanmtl.com but using vpn it was avoided but snice some time the speed of crawling has been slowed down to extreme

that is most likely that the server sends an HTTP 429 and asks WebToEpub to slow down, which WebToEpub does. If WebToEpub ignores this it could be that your ip gets banned etc.

Your vpn could be the reason that it is slowed down. Why? A lot of users get the same ip for the internet and cloudflare realized this one ip address generates a lot of traffic additionally if multiple people scrape the internet with this ip cloudflare knows it is a bot and gets aggressive with 403 are you a human.

@sksmsalman it could be that you are requesting to many chapters from the server in a short time span try this option Manual Throttle|Delay per chapter Can you give more context? What is your browser and version?

sksmsalman commented 1 week ago

Same happened to fanmtl.com but using vpn it was avoided but snice some time the speed of crawling has been slowed down to extreme

that is most likely that the server sends an HTTP 429 and asks WebToEpub to slow down, which WebToEpub does. If WebToEpub ignores this it could be that your ip gets banned etc.

Your vpn could be the reason that it is slowed down. Why? A lot of users get the same ip for the internet and cloudflare realized this one ip address generates a lot of traffic additionally if multiple people scrape the internet with this ip cloudflare knows it is a bot and gets aggressive with 403 are you a human.

@sksmsalman it could be that you are requesting to many chapters from the server in a short time span try this option Manual Throttle|Delay per chapter Can you give more context? What is your browser and version?

Hmm My browser is Kiwi Browser 124.0.6327.4 Also used in Yandex 126.0.6478.41 (64-bit version) Also used in Firefox nightly 132.0a1 (Build #2016042479), All are used in android Tried with and without vpn (touch vpn extension, secure vpn, provido vpn were used through different browser with different time interval)

gamebeaker commented 1 week ago

@sksmsalman as a short time fix normal firefox android works for me (i don't think kiwi will get fix in the short term). I think it is related to #1447 but that the compatibility list https://developer.mozilla.org/en-US/docs/Web/Privacy/Privacy_sandbox/Partitioned_cookies isn't updated and that firefox android is Nightly. image

gamebeaker commented 1 week ago

@sksmsalman does normal firefox android work for you? edit: opened issue for mdn: here