knownsec / LSpider

LSpider 一个为被动扫描器定制的前端爬虫
MIT License
345 stars 54 forks source link

docker搭建后,在lspider的docker环境中执行,如下报错,请大佬告知一下,什么原因 #19

Open uunnsec opened 3 years ago

uunnsec commented 3 years ago
/opt/LSpider # python3 manage.py SpiderCoreBackendStart --test
[INFO] [MainThread] [03:55:17] [SpiderCoreBackendStart.py:35] [Spider] start test spider.
[INFO] [MainThread] [03:55:17] [rabbitmqhandler.py:39] [Monitor][INIT][Rabbitmq] New Rabbitmq link to rabbitmq
[INFO] [MainThread] [03:55:17] [rabbitmqhandler.py:36] [Monitor][INIT] Rabbitmq init success...
[INFO] [MainThread] [03:55:17] [chromeheadless.py:100] [Chrome Headless] Proxy 127.0.0.1:7777 init
[ERROR] [MainThread] [03:55:17] [chromeheadless.py:45] [Chrome Headless] ChromeDriver load error.
[ERROR] [MainThread] [03:55:17] [SpiderCoreBackendStart.py:47] [Spider] something error, Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
    stdin=PIPE)
  File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/opt/LSpider/bin/chromedriver': '/opt/LSpider/bin/chromedriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/LSpider/core/chromeheadless.py", line 38, in __init__
    self.init_object()
  File "/opt/LSpider/core/chromeheadless.py", line 119, in init_object
    desired_capabilities=desired_capabilities)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    self.service.start()
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/LSpider/web/spider/management/commands/SpiderCoreBackendStart.py", line 40, in handle
    spidercore = SpiderCore(test_target_list)
  File "/opt/LSpider/web/spider/controller/spider.py", line 239, in __init__
    self.req = LReq(is_chrome=True)
  File "/opt/LSpider/utils/LReq.py", line 37, in __init__
    self.cs = ChromeDriver()
  File "/opt/LSpider/core/chromeheadless.py", line 46, in __init__
    exit(0)
  File "/usr/local/lib/python3.7/_sitebuiltins.py", line 26, in __call__
    raise SystemExit(code)
SystemExit: 0

Exception ignored in: <function ChromeDriver.__del__ at 0x7f91f2b63680>
Traceback (most recent call last):
  File "/opt/LSpider/core/chromeheadless.py", line 591, in __del__
    self.close_driver()
  File "/opt/LSpider/core/chromeheadless.py", line 586, in close_driver
    self.driver.quit()
AttributeError: 'ChromeDriver' object has no attribute 'driver'
LoRexxar commented 3 years ago

没有按照配置要求配置chromewebdriver

No such file or directory: '/opt/LSpider/bin/chromedriver': '/opt/LSpider/bin/chromedriver'

uunnsec commented 3 years ago

没有按照配置要求配置chromewebdriver

没有这样的文件或目录:'/opt/LSpider/bin/chromedriver':'/opt/LSpider/bin/chromedriver'

多谢大佬,我直接用你项目,使用docker-compose构建的 根据上面报错情况,增加以下配置后,可以解决这个问题,大佬应该将这个配置加到docker-compose中 mkdir /opt/LSpider/bin/ ln -s /usr/lib/chromium/chromedriver /opt/LSpider/bin/chromedriver

LoRexxar commented 3 years ago

这个chromedriver的路径应该不是固定的,我研究研究