dipu-bd / lightnovel-crawler

Generate and download e-books from online sources.
https://pypi.org/project/lightnovel-crawler/
GNU General Public License v3.0
1.43k stars 279 forks source link

Fix this source Wuxiaworld.com #2170

Closed krebsf01 closed 6 months ago

krebsf01 commented 10 months ago

Let us know

Novel URL: https://www.wuxiaworld.com/novel/overgeared App Location: EXE App Version: 3.3.1

Describe this issue

When trying to scrap from Wuxiaworld I get the following error

Traceback (most recent call last):
  File "lncrawl\templates\browser\basic.py", line 88, in read_novel_info
  File "C:\Users\THIAGO~1.KRE\AppData\Local\Temp\_MEI48202\sources\en\w\wuxiacom.py", line 38, in read_novel_info_in_scraper
    response = self.grpc.request(
  File "pyease_grpc\rpc_session.py", line 150, in request
  File "requests\models.py", line 1021, in raise_for_status
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api2.wuxiaworld.com/wuxiaworld.api.v2.Novels/GetNovel

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "threading.py", line 1016, in _bootstrap_inner
  File "threading.py", line 953, in run
  File "lncrawl\templates\browser\basic.py", line 95, in read_novel_info
  File "C:\Users\THIAGO~1.KRE\AppData\Local\Temp\_MEI48202\sources\en\w\wuxiacom.py", line 149, in read_novel_info_in_browser
    self.visit(self.novel_url)
  File "lncrawl\templates\browser\basic.py", line 65, in visit
  File "lncrawl\core\browser.py", line 155, in visit
  File "lncrawl\core\browser.py", line 62, in _init_browser
  File "lncrawl\webdriver\__init__.py", line 35, in create_new
  File "lncrawl\webdriver\local.py", line 115, in create_local
  File "undetected_chromedriver\__init__.py", line 398, in __init__
AttributeError: 'ChromeOptions' object has no attribute 'headless'

 ! Error: No chapters found
<class 'Exception'>
File "lncrawl\bots\console\integration.py", line 107, in start
    raise e
  File "lncrawl\bots\console\integration.py", line 101, in start
    _download_novel()
  File "lncrawl\bots\console\integration.py", line 85, in _download_novel
    self.app.get_novel_info()
  File "lncrawl\core\app.py", line 137, in get_novel_info
    raise Exception("No chapters found") 
alzamer2 commented 8 months ago

hello new update was issued for wuxiaworld.com update your sources and try scraping

alzamer2 commented 8 months ago

hi from

AttributeError: 'ChromeOptions' object has no attribute 'headless'

it seem this error not with wuxiaworld.com but with headless error check #2211

by the way i'm using undetected_chromedriver-3.5.4 webdriver_manager-4.0.0 selenium-4.11.2 Chrome Version 120.0.6099.72

the lightnovel-crawler is working what your version?