SpiderClub / weibospider

:zap: A distributed crawler for weibo, building with celery and requests.
MIT License
4.81k stars 1.21k forks source link

安装依赖lib的时候 celery==4.1.0报错 #183

Closed logie closed 4 years ago

logie commented 4 years ago

在提交Issue之前请先回答下面问题,谢谢!

1.你是怎么操作的? jiayantao@ubuntu:~$ sudo pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple celery==4.1.0 [sudo] password for jiayantao:

The directory '/home/jiayantao/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/home/jiayantao/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Requirement already satisfied (use --upgrade to upgrade): celery==4.1.0 in /usr/local/lib/python3.5/site-packages Requirement already satisfied (use --upgrade to upgrade): kombu<5.0,>=4.0.2 in /usr/local/lib/python3.5/site-packages (from celery==4.1.0) Requirement already satisfied (use --upgrade to upgrade): pytz>dev in /usr/local/lib/python3.5/site-packages (from celery==4.1.0) Requirement already satisfied (use --upgrade to upgrade): billiard<3.6.0,>=3.5.0.2 in /usr/local/lib/python3.5/site-packages (from celery==4.1.0) Requirement already satisfied (use --upgrade to upgrade): importlib-metadata>=0.18 in /usr/local/lib/python3.5/site-packages (from kombu<5.0,>=4.0.2->celery==4.1.0) Requirement already satisfied (use --upgrade to upgrade): amqp<2.6,>=2.5.2 in /usr/local/lib/python3.5/site-packages (from kombu<5.0,>=4.0.2->celery==4.1.0) Collecting zipp>=0.5 (from importlib-metadata>=0.18->kombu<5.0,>=4.0.2->celery==4.1.0) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/60/85/668bca4a9ef474ca634c993e768f12bd99af1f06bb90bb2655bc538a967e/zipp-2.2.0.tar.gz

Running setup.py (path:/tmp/pip-build-t843ka12/zipp/setup.py) egg_info for package zipp produced metadata for project name unknown. Fix your #egg=zipp fragments.

Requirement already satisfied (use --upgrade to upgrade): unknown from https://pypi.tuna.tsinghua.edu.cn/packages/60/85/668bca4a9ef474ca634c993e768f12bd99af1f06bb90bb2655bc538a967e/zipp-2.2.0.tar.gz#sha256=5c56e330306215cd3553342cfafc73dda2c60792384117893f3a83f8a1209f50 in /usr/local/lib/python3.5/site-packages (from importlib-metadata>=0.18->kombu<5.0,>=4.0.2->celery==4.1.0) Requirement already satisfied (use --upgrade to upgrade): vine<5.0.0a1,>=1.1.3 in /usr/local/lib/python3.5/site-packages (from amqp<2.6,>=2.5.2->kombu<5.0,>=4.0.2->celery==4.1.0) You are using pip version 8.1.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

在执行celery -A tasks.workers -Q login_queue,user_crawler,fans_followers,search_crawler,home_crawler worker -l info -c 1 报错

`jiayantao@ubuntu:~$ celery -A tasks.workers -Q login_queue,user_crawler,fans_followers,search_crawler,home_crawler worker -l info -c 1 Traceback (most recent call last): File "/usr/local/lib/python3.5/site-packages/kombu/utils/compat.py", line 12, in from importlib import metadata as importlib_metadata ImportError: cannot import name 'metadata'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/celery", line 11, in sys.exit(main()) File "/usr/local/lib/python3.5/site-packages/celery/main.py", line 13, in main from celery.bin.celery import main as _main File "/usr/local/lib/python3.5/site-packages/celery/bin/init.py", line 2, in from .base import Option File "/usr/local/lib/python3.5/site-packages/celery/bin/base.py", line 17, in from celery import VERSION_BANNER, Celery, maybe_patch_concurrency File "/usr/local/lib/python3.5/site-packages/celery/local.py", line 509, in getattr module = import(self._object_origins[name], None, None, [name]) File "/usr/local/lib/python3.5/site-packages/celery/app/init.py", line 5, in from celery import _state File "/usr/local/lib/python3.5/site-packages/celery/_state.py", line 15, in from celery.utils.threads import LocalStack File "/usr/local/lib/python3.5/site-packages/celery/utils/init.py", line 9, in from .functional import memoize # noqa File "/usr/local/lib/python3.5/site-packages/celery/utils/functional.py", line 11, in from kombu.utils.functional import ( File "/usr/local/lib/python3.5/site-packages/kombu/utils/init.py", line 5, in from .compat import fileno, maybe_fileno, nested, register_after_fork File "/usr/local/lib/python3.5/site-packages/kombu/utils/compat.py", line 14, in import importlib_metadata File "/usr/local/lib/python3.5/site-packages/importlib_metadata/init.py", line 9, in import zipp ImportError: No module named 'zipp'`

2.你期望的结果是什么?

3.实际上你得到的结果是什么?

4.你使用的是哪个版本的WeiboSpider? 你的操作系统是什么?是否有读本项目的[常见问题] 1.7.2 release版本 ubuntu 16.04.6

thekingofcity commented 4 years ago

建议使用celery==4.2.1和kombu==4.2.1

OneCodeMonkey commented 4 years ago

@logie 好像是 kombu 包的冲突,我用的是 celery==4.3.0 && kombu==4.6.4 ,可以试下

logie commented 4 years ago

我先安装的celery==4.3.0 && kombu==4.6.4 ,还是不能启动, 又重新先安装kombu==4.2.1,再安装celery==4.2.1 可以成功运行启动了 感谢二位的解答

thekingofcity commented 4 years ago

@logie 好像是 kombu 包的冲突,我用的是 celery==4.3.0 && kombu==4.6.4 ,可以试下

@OneCodeMonkey 你这个到底行不行呀(逃