binux / pyspider

A Powerful Spider(Web Crawler) System in Python.
http://docs.pyspider.org/
Apache License 2.0
16.43k stars 3.69k forks source link

Which python3 version do you support? #217

Closed facat closed 9 years ago

facat commented 9 years ago
except beanstalkc.CommandFailed, err:

This exception statement in message_queue/beanstalk.py seems to only work in python2.

And I cannot run pyspider in python3.4. Below is racetrack information.

Traceback (most recent call last):
  File "/mnt/250g/down/document/python/soufang2/bin/pyspider", line 9, in <module>
    load_entry_point('pyspider==0.3.6', 'console_scripts', 'pyspider')()
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/run.py", line 725, in main
    cli()
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 971, in invoke
    return Command.invoke(self, ctx)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/run.py", line 159, in cli
    ctx.invoke(all)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/run.py", line 475, in all
    ctx.invoke(webui, **webui_config)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/run.py", line 306, in webui
    app = load_cls(None, None, webui_instance)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/run.py", line 47, in load_cls
    return utils.load_object(value)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/libs/utils.py", line 348, in load_object
    module = __import__(module_name, globals(), locals(), [object_name])
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/webui/__init__.py", line 8, in <module>
    from . import app, index, debug, task, result, login
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/pyspider-0.3.6-py3.4.egg/pyspider/webui/app.py", line 89, in <module>
    template_folder=os.path.join(os.path.dirname(__file__), 'templates'))
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/flask/app.py", line 319, in __init__
    template_folder=template_folder)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/flask/helpers.py", line 741, in __init__
    self.root_path = get_root_path(self.import_name)
  File "/mnt/250g/down/document/python/soufang2/lib/python3.4/site-packages/flask/helpers.py", line 631, in get_root_path
    loader = pkgutil.get_loader(import_name)
  File "/home/cubie/app/python3/lib/python3.4/pkgutil.py", line 467, in get_loader
    return find_loader(fullname)
  File "/home/cubie/app/python3/lib/python3.4/pkgutil.py", line 488, in find_loader
    return spec.loader
AttributeError: 'NoneType' object has no attribute 'loader'

Which version of python3 shoudl I use?

binux commented 9 years ago

beanstalkc doesn't support python 3. flask: https://github.com/binux/pyspider/issues/194

facat commented 9 years ago

@binux What should I do? Use python2 instead?

binux commented 9 years ago

upgrade to python 3.4.1 may deal with this bug.

facat commented 9 years ago

@binux There is no python3.4.1. I'll try python3.4.2. Before that, I have tried python3.5 beta1 with no luck.

binux commented 9 years ago

the travis-ci is using python 3.4.2, I believe it works. I hadn't tested py3.5 yet,

facat commented 9 years ago

@binux Works under python3.4.2. Don't know why.