[Solved]Issues while extracting from bookwalker

[ann840925]

Sorry, everybody Why does my program just stop here

[ann840925]

[xuzhengyi1995]

Hello,

This is very strange, you can check the #30 , you got exactly the same problem, and you are also using a chrome 89.0.4389.114

Now I think maybe this is a problem of Chrome driver version, you could try to check your Chrome version, and make sure it can work with your Chrome driver, and you can just redownload it from here

[ulyssesric]

Have exactly the same problem here. ested in Python 3.8.8 and 3.9.2. Same result.

Seems like a conflict between Chrome Driver version 89.0.4389.23 and Chrome version 89.0.4389.114. Either Command.GET just don't work as expected, or don't give any response event. I've tried to run the process while monitoring processes, and I found that Google Chrome did started to do the task, as a subprocess "Google Chrome Helper (Renderer)" will be created, but that subprocess eats up all the CPU (96.9% on my computer) and can't be terminated. Didn't have this problem with Google Chrome previous release.

Unfortunately there isn't another Chrome Driver release for v89 as they jumped to Chrome V90 by now, and Google Chrome v90 is not officially released yet, and it's not possible to downgrade Chrome to previous release.

BTW, Chrome Driver v90 won't work with 89.0.4389.114. I've tried.

[xuzhengyi1995]

Have exactly the same problem here. ested in Python 3.8.8 and 3.9.2. Same result.

Seems like a conflict between Chrome Driver version 89.0.4389.23 and Chrome version 89.0.4389.114. Either Command.GET just don't work as expected, or don't give any response event. I've tried to run the process while monitoring processes, and I found that Google Chrome did started to do the task, as a subprocess "Google Chrome Helper (Renderer)" will be created, but that subprocess eats up all the CPU (96.9% on my computer) and can't be terminated. Didn't have this problem with Google Chrome previous release.

Unfortunately there isn't another Chrome Driver release for v89 as they jumped to Chrome V90 by now, and Google Chrome v90 is not officially released yet, and it's not possible to downgrade Chrome to previous release.

BTW, Chrome Driver v90 won't work with 89.0.4389.114. I've tried.

Hello,

Thank you for your reply, I think maybe there is a compatibility problem of Chrome and ChromeDriver, I will try it and try to find out the solution.

Thank you!

[Pepsyy]

I tried with firefox, and got the same problem :/ But I just changed the cookie and launched gecko at the same time, so maybe something more is needed to use selinium with Firefox

[xuzhengyi1995]

I tried with firefox, and got the same problem :/ But I just changed the cookie and launched gecko at the same time, so maybe something more is needed to use selinium with Firefox

Hello,

I have tried and I find that when chrome is loading the manga page, it just hang on and does not response to any thing, even I tried to run some console.log(1) on the console, there is no response at all, and when I try to close the browser, it just hang on and can not be closed.

I check the network packet and the HTTP request and response, everything is fine, the data has transferred but the Chrome just keep no response.

This is very strange and I have really no idea what happened, never seen something like this before. Maybe some JavaScript blocked the Chrome? I have no idea.

I tried to load some other page (like google), and they can load normally.

If you have any idea, please tell me, thank you!

Thank you

[xuzhengyi1995]

I think it's the JavaScript of BW, the getLoader js, it blocked the Chrome, because I just copied the whole script to the normal Chrome, it just give an error, but to the Chrome opened with Chrome driver, it just blocked.

[xuzhengyi1995]

I have check that they have add the code to detect webdriver (window.navigator), I will updated the code soon, please try again, thank you!

[xuzhengyi1995]

Hello,

I have changed the driver to undetected_chromedriver, I have tried on my computer and it can load the page now.

Please pull the new code and install undetected_chromedriver with pip

pip install undetected_chromedriver

And then try again, thank you!

[xuzhengyi1995]

Downloaded successfully, please try it, thank you !

[ann840925]

Thanks for xuzhengyi And sorry, I still have the same problem with fix version My chromedriver is 89.0.4389.23 My chrome is 89.0.4389.114

By the way... 'Bloom into you' is perfect ><

[xuzhengyi1995]

Yes it's perfect :)

Then I think maybe BW JP has something different with BW TW, I will check it later.

Thank you for your patient!

[xuzhengyi1995]

I have just found out that it could detect headless mode, I just disable it temporarily, I will try to figure it out in the future.

[ann840925]

There is the same problem in my BW TW. Maybe just my wrong ; ;

[ann840925]

I successfully download it! Thank you very much!!

[xuzhengyi1995]

Now I think it should work, I checked on my computer and it can work in headless mode and for both JP and TW.

The manga just for test on BW JP:

If it still not work on your computer, I...for now I really have no idea what is going wrong...maybe I can do more test tomorrow, and we can see that maybe other people can download it.

Really thank you for your test.

[xuzhengyi1995]

I successfully download it! Thank you very much!!

Great! Thank you for your patient!