search

binux / pyspider

A Powerful Spider(Web Crawler) System in Python.
http://docs.pyspider.org/
Apache License 2.0
16.48k stars 3.69k forks source link

NameError: name 'BaseHandler' is not defined #969

Open SJshenjian opened 3 years ago

SJshenjian commented 3 years ago

C:\Users\user.conda\envs\pyspider\python.exe "E:\Program Files\JetBrains\PyCharm 2020.3.3\plugins\python\helpers\pydev\pydevd.py" --multiproc --qt-support=auto --client 127.0.0.1 --port 55792 --file E:/Project/pyspider/run.py -c config_dev.json all Connected to pydev debugger (build 203.7148.72) E:\Project\pyspider\pyspider\libs\utils.py:196: FutureWarning: timeout is not supported on your platform. warnings.warn("timeout is not supported on your platform.", FutureWarning) [W 210423 09:58:50 run:427] phantomjs not found, continue running without it. [W 210423 09:58:52 run:470] puppeteer not found, continue running without it. C:\Users\user.conda\envs\pyspider\lib\site-packages\amqp\connection.py:321: AMQPDeprecationWarning: The .frame_writer attribute on the connection was accessed before the connection was established. This is supported for now, but will be deprecated in amqp 2.2.0.

Since amqp 2.0 you have to explicitly call Connection.connect() before using the connection.

W_FORCE_CONNECT.format(attr=attr))) [I 210423 09:58:55 result_worker:50] result_worker starting... [I 210423 09:58:56 tornado_fetcher:748] fetcher starting... [I 210423 09:58:56 processor:211] processor starting... [I 210423 09:59:01 scheduler:691] scheduler starting... [I 210423 09:59:01 scheduler:836] scheduler.xmlrpc listening on 127.0.0.1:23333 [I 210423 09:59:01 scheduler:127] project lianjiachengjiao updated, status:STOP, paused:False, 0 tasks [I 210423 09:59:01 scheduler:127] project paimai_jd updated, status:STOP, paused:False, 0 tasks [I 210423 09:59:01 scheduler:127] project sell_beike updated, status:RUNNING, paused:False, 0 tasks [I 210423 09:59:01 scheduler:127] project test4 updated, status:TODO, paused:False, 0 tasks [I 210423 09:59:01 scheduler:127] project test updated, status:STOP, paused:False, 0 tasks [I 210423 09:59:01 scheduler:127] project sell_zhuge updated, status:STOP, paused:False, 0 tasks [I 210423 09:59:01 scheduler:1017] select sell_beike:_on_get_info data:,_on_get_info [I 210423 09:59:01 scheduler:127] project test7 updated, status:TODO, paused:False, 0 tasks [I 210423 09:59:07 tornado_fetcher:781] fetcher exiting... [E 210423 09:59:07 rabbitmq:46] RabbitMQ error: BlockingIOError(10035, '无法立即完成一个非阻止性套接字操作。', None, 10035, None), reconnect. [E 210423 09:59:08 scheduler:1275] [WinError 10035] 无法立即完成一个非阻止性套接字操作。 Traceback (most recent call last): File "E:\Project\pyspider\pyspider\message_queue\rabbitmq.py", line 44, in wrap return func(self, *args, **kwargs) File "E:\Project\pyspider\pyspider\message_queue\rabbitmq.py", line 256, in put_nowait return self.channel.basic_publish(msg, exchange="", routing_key=self.name) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\amqp\channel.py", line 1758, in _basic_publish self.connection.drain_events(timeout=0) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\amqp\connection.py", line 500, in drain_events while not self.blocking_read(timeout): File "C:\Users\user.conda\envs\pyspider\lib\site-packages\amqp\connection.py", line 505, in blocking_read frame = self.transport.read_frame() File "C:\Users\user.conda\envs\pyspider\lib\site-packages\amqp\transport.py", line 250, in read_frame frame_header = read(7, True) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\amqp\transport.py", line 430, in _read s = recv(n - len(rbuf)) BlockingIOError: [WinError 10035] 无法立即完成一个非阻止性套接字操作。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Project\pyspider\pyspider\scheduler\scheduler.py", line 1273, in _thread_worker
    method(*args, **kwargs)
  File "E:\Project\pyspider\pyspider\scheduler\scheduler.py", line 260, in _update_project
    'callback': '_on_get_info',
  File "E:\Project\pyspider\pyspider\scheduler\scheduler.py", line 1031, in on_select_task
    self.send_task(task)
  File "E:\Project\pyspider\pyspider\scheduler\scheduler.py", line 357, in send_task
    self.out_queue.put_nowait(task)
  File "E:\Project\pyspider\pyspider\message_queue\rabbitmq.py", line 48, in wrap
    return func(self, *args, **kwargs)
  File "E:\Project\pyspider\pyspider\message_queue\rabbitmq.py", line 256, in put_nowait
    return self.channel.basic_publish(msg, exchange="", routing_key=self.name)
  File "C:\Users\user\.conda\envs\pyspider\lib\site-packages\amqp\channel.py", line 1758, in _basic_publish
    self.connection.drain_events(timeout=0)
  File "C:\Users\user\.conda\envs\pyspider\lib\site-packages\amqp\connection.py", line 500, in drain_events
    while not self.blocking_read(timeout):
  File "C:\Users\user\.conda\envs\pyspider\lib\site-packages\amqp\connection.py", line 505, in blocking_read
    frame = self.transport.read_frame()
  File "C:\Users\user\.conda\envs\pyspider\lib\site-packages\amqp\transport.py", line 250, in read_frame
    frame_header = read(7, True)
  File "C:\Users\user\.conda\envs\pyspider\lib\site-packages\amqp\transport.py", line 430, in _read
    s = recv(n - len(rbuf))
BlockingIOError: [WinError 10035] 无法立即完成一个非阻止性套接字操作。

[I 210423 09:59:08 scheduler:127] project test1 updated, status:TODO, paused:False, 0 tasks [I 210423 09:59:11 processor:229] processor exiting... [I 210423 09:59:11 result_worker:67] result_worker exiting... [I 210423 09:59:13 scheduler:630] in 5m: new:0,success:0,retry:0,failed:0 [I 210423 09:59:13 scheduler:708] scheduler exiting... Traceback (most recent call last): File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\core.py", line 696, in main rv = self.invoke(ctx) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\core.py", line 1060, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\core.py", line 889, in invoke return ctx.invoke(self.callback, ctx.params) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\core.py", line 534, in invoke return callback(*args, *kwargs) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\decorators.py", line 17, in new_func return f(get_current_context(), args, kwargs) File "E:\Project\pyspider\pyspider\run.py", line 563, in all ctx.invoke(webui, webui_config) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\core.py", line 534, in invoke return callback(*args, *kwargs) File "C:\Users\user.conda\envs\pyspider\lib\site-packages\click\decorators.py", line 17, in new_func return f(get_current_context(), args, kwargs) File "E:\Project\pyspider\pyspider\run.py", line 344, in webui app = load_cls(None, None, webui_instance) File "E:\Project\pyspider\pyspider\run.py", line 48, in load_cls return utils.load_object(value) File "E:\Project\pyspider\pyspider\libs\utils.py", line 369, in load_object module = import(module_name, globals(), locals(), [object_name]) File "E:\Project\pyspider\pyspider\webui__init__.py", line 8, in from . import app, index, debug, task, result, login File "E:\Project\pyspider\pyspider\webui\debug.py", line 22, in from pyspider.libs import utils, sample_handler, dataurl File "E:\Project\pyspider\pyspider\libs\sample_handler.py", line 9, in class Handler(BaseHandler): NameError: name 'BaseHandler' is not defined

SJshenjian commented 3 years ago

I run it with -c config_dev.json all mode, I want to know how to solve it, thanks