dataabc / weiboSpider

新浪微博爬虫,用python爬取新浪微博数据
8.15k stars 1.95k forks source link

请问在weiboSpider目录运行 python3 -m weibo_spider 的时候没反应,没有在当前目录创建config.json配置文件是什么问题? #516

Closed xpywhq closed 1 year ago

xpywhq commented 1 year ago

为了更好的解决问题,请认真回答下面的问题。等到问题解决,请及时关闭本issue。

答:源码安装的

答:我前期在Git Bash中运行安装的,相关log如下

Xing Poyaun@LAPTOP-0HTFA64B MINGW64 /d/Users/Xing Poyaun/GitDatabase (master) $ git clone https://github.com/dataabc/weiboSpider.git Cloning into 'weiboSpider'... remote: Enumerating objects: 1680, done. remote: Counting objects: 100% (255/255), done. remote: Compressing objects: 100% (137/137), done. remote: Total 1680 (delta 144), reused 176 (delta 118), pack-reused 1425 Receiving objects: 100% (1680/1680), 574.85 KiB | 523.00 KiB/s, done. Resolving deltas: 100% (940/940), done.

Xing Poyaun@LAPTOP-0HTFA64B MINGW64 /d/Users/Xing Poyaun/GitDatabase (master) $ cd weiboSpider

Xing Poyaun@LAPTOP-0HTFA64B MINGW64 /d/Users/Xing Poyaun/GitDatabase/weiboSpider (master) $ pip install -r requirements.txt Requirement already satisfied: lxml==4.9.1 in d:\softwares\programming_softwares\anaconda3\lib\site-packages (from -r requirements.txt (line 1)) (4.9.1) Collecting requests==2.23.0 Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB) -------------------------------------- 58.4/58.4 kB 140.0 kB/s eta 0:00:00 Collecting tqdm==4.46.1 Downloading tqdm-4.46.1-py2.py3-none-any.whl (63 kB) -------------------------------------- 63.7/63.7 kB 568.4 kB/s eta 0:00:00 Collecting absl-py==0.12.0 Downloading absl_py-0.12.0-py3-none-any.whl (129 kB) ------------------------------------ 129.4/129.4 kB 763.7 kB/s eta 0:00:00 Collecting idna<3,>=2.5 Downloading idna-2.10-py2.py3-none-any.whl (58 kB) ---------------------------------------- 58.8/58.8 kB 1.0 MB/s eta 0:00:00 Collecting chardet<4,>=3.0.2 Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB) -------------------------------------- 133.4/133.4 kB 1.3 MB/s eta 0:00:00 Requirement already satisfied: certifi>=2017.4.17 in d:\softwares\programming_softwares\anaconda3\lib\site-packages (from requests==2.23.0->-r requirements.txt (line 2)) (2022.9.14) Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB) -------------------------------------- 128.0/128.0 kB 1.5 MB/s eta 0:00:00 Requirement already satisfied: six in d:\softwares\programming_softwares\anaconda3\lib\site-packages (from absl-py==0.12.0->-r requirements.txt (line 4)) (1.16.0) Installing collected packages: chardet, urllib3, tqdm, idna, absl-py, requests Attempting uninstall: chardet Found existing installation: chardet 4.0.0 Uninstalling chardet-4.0.0: Successfully uninstalled chardet-4.0.0 Attempting uninstall: urllib3 Found existing installation: urllib3 1.26.11 Uninstalling urllib3-1.26.11: Successfully uninstalled urllib3-1.26.11 Attempting uninstall: tqdm Found existing installation: tqdm 4.64.1 Uninstalling tqdm-4.64.1: Successfully uninstalled tqdm-4.64.1 Attempting uninstall: idna Found existing installation: idna 3.3 Uninstalling idna-3.3: Successfully uninstalled idna-3.3 Attempting uninstall: requests Found existing installation: requests 2.28.1 Uninstalling requests-2.28.1: Successfully uninstalled requests-2.28.1 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. spyder 5.2.2 requires pyqt5<5.13, which is not installed. spyder 5.2.2 requires pyqtwebengine<5.13, which is not installed. panel 0.13.1 requires tqdm>=4.48.0, but you have tqdm 4.46.1 which is incompatible. conda-repo-cli 1.0.20 requires clyent==1.2.1, but you have clyent 1.2.2 which is incompatible. conda-repo-cli 1.0.20 requires nbformat==5.4.0, but you have nbformat 5.5.0 which is incompatible. conda-repo-cli 1.0.20 requires requests==2.28.1, but you have requests 2.23.0 which is incompatible. anaconda-client 1.11.0 requires tqdm>=4.56.0, but you have tqdm 4.46.1 which is incompatible. anaconda-client 1.11.0 requires urllib3>=1.26.4, but you have urllib3 1.25.11 which is incompatible. Successfully installed absl-py-0.12.0 chardet-3.0.4 idna-2.10 requests-2.23.0 tqdm-4.46.1 urllib3-1.25.11

Xing Poyaun@LAPTOP-0HTFA64B MINGW64 /d/Users/Xing Poyaun/GitDatabase/weiboSpider (master) 对照spyder 5.22 需要的库,我在pycharm中对应下载了最新版本的,因为第一个库要求版本不能安装上。所以索性全部所需库安装了最新版的。

这是我运行了python3 -m weibo_spider后没有反应的文件夹截图

答:

答:

答:

答:

dataabc commented 1 year ago

pycharm运行有时可能会出错,您使用cmd工具运行看看。

xpywhq commented 1 year ago

image 大佬,还是不行

dataabc commented 1 year ago

您可以单独运行main.py,效果一样。

xpywhq commented 1 year ago

大佬师,我直接单独运行main.py。结果:如图1所示,是否要更换urllib3和chardet的版本;如图2所示,文件夹中新增了config.json等文件,是否可以直接尝试配置config.json使用,谢谢。 image image

dataabc commented 1 year ago

不用换,直接配置config就可以了。

xpywhq commented 1 year ago

大佬师,继续运行下面的命令python3 -m weibo_spider --config_path="config.json"没有任何变化,以下是图示。是否自行在config.json配置相关参数即可?谢谢 image

dataabc commented 1 year ago

运行main.py

xpywhq commented 1 year ago

大佬师,可以运行了,谢谢您耐心的指导。 image

xpywhq commented 1 year ago

谢谢

dataabc commented 1 year ago

不客气。

xpywhq commented 1 year ago

大佬,复盘了下之前的操作,还想请教您一下:为何cmd直接调用python3 -m weibo_spider等的命令都不work,再次感谢。

dataabc commented 1 year ago

这个我也不清楚,可能软件环境没配置好。

xpywhq commented 1 year ago

谢谢

---- 回复的原邮件 ---- | 发件人 | Chen @.> | | 日期 | 2023年04月03日 00:49 | | 收件人 | @.> | | 抄送至 | @.>@.> | | 主题 | Re: [dataabc/weiboSpider] 请问在weiboSpider目录运行 python3 -m weibo_spider 的时候没反应,没有在当前目录创建config.json配置文件是什么问题? (Issue #516) |

这个我也不清楚,可能软件环境没配置好。

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 1 year ago

Closing as stale, please reopen if you'd like to work on this further.