search

schemen / m2em

Manga to eManga
MIT License
51 stars 6 forks source link

Selenium headless fix #20

Closed eipind closed 5 years ago

eipind commented 5 years ago

I should've included selenium in the requirements.txt file, will do soon

schemen commented 5 years ago

I want to work this in the fix_mangafox branch - do you have any potential idea if the selenium library is faster that requests-html? I am currently heavily sick, I will have a closer look as soon as I can.

eipind commented 5 years ago

Hope you get better soon :) I'll take a look at it this week, both requests-html and scraper. If you'd like I'll make a PR for either of them and you can test it yourself when you have the time

eipind commented 5 years ago

Just to let you know, I've been trying out scrapy and it looks like I'll need to use Splash with it for javascript, unless we with go with simulating the AJAX requests as per your suggestion. I'll mess around with it more and next week I'll try the requests-html library you suggested.

schemen commented 5 years ago

If we end up scrapy-fying mangafox, I might as well open a branch to scrapy-fy m2em I think.

And thanks for the research! Recovery is still going on but better :)

schemen commented 5 years ago

@eipind I will close this PR for now, please have a look at #18. If you want, you can review the "new" setup, let me know if anything is unclear :)