search

dataabc / weibo-crawler

新浪微博爬虫，用python爬取新浪微博数据，并下载微博图片和微博视频
3.25k stars 739 forks source link

安装依赖时的问题 #384

Open MystFeng opened 1 year ago

MystFeng commented 1 year ago

安装依赖没装完，不知道哪里有问题，研究了好久都没弄懂，我该做什么？ 这是控制台提示 C:\Users\delic\Documents\GitHub\weibo-crawler>pip install -r requirements.txt Collecting lxml==4.9.1 (from -r requirements.txt (line 1)) Using cached lxml-4.9.1.tar.gz (3.4 MB) Preparing metadata (setup.py) ... done Requirement already satisfied: pymongo==3.7.2 in c:\python311\lib\site-packages (from -r requirements.txt (line 2)) (3.7.2) Requirement already satisfied: pymysql==1.0.2 in c:\python311\lib\site-packages (from -r requirements.txt (line 3)) (1.0.2) Requirement already satisfied: requests==2.31.0 in c:\python311\lib\site-packages (from -r requirements.txt (line 4)) (2.31.0) Collecting tqdm==4.64.0 (from -r requirements.txt (line 5)) Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB) Requirement already satisfied: charset-normalizer<4,>=2 in c:\python311\lib\site-packages (from requests==2.31.0->-r requirements.txt (line 4)) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in c:\python311\lib\site-packages (from requests==2.31.0->-r requirements.txt (line 4)) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in c:\python311\lib\site-packages (from requests==2.31.0->-r requirements.txt (line 4)) (2.0.3) Requirement already satisfied: certifi>=2017.4.17 in c:\python311\lib\site-packages (from requests==2.31.0->-r requirements.txt (line 4)) (2023.5.7) Collecting colorama (from tqdm==4.64.0->-r requirements.txt (line 5)) Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB) Building wheels for collected packages: lxml Building wheel for lxml (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [94 lines of output] Building lxml version 4.9.1. Building without Cython. Building against pre-built libxml2 andl libxslt libraries running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-cpython-311 creating build\lib.win-amd64-cpython-311\lxml copying src\lxml\builder.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\cssselect.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\doctestcompare.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\ElementInclude.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\pyclasslookup.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\sax.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\usedoctest.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml_elementpath.py -> build\lib.win-amd64-cpython-311\lxml copying src\lxml__init.py -> build\lib.win-amd64-cpython-311\lxml creating build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes__init__.py -> build\lib.win-amd64-cpython-311\lxml\includes creating build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\builder.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\clean.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\defs.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\diff.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\ElementSoup.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\formfill.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\html5parser.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\soupparser.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\usedoctest.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html_diffcommand.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html_html5builder.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html_setmixin.py -> build\lib.win-amd64-cpython-311\lxml\html copying src\lxml\html\init.py -> build\lib.win-amd64-cpython-311\lxml\html creating build\lib.win-amd64-cpython-311\lxml\isoschematron copying src\lxml\isoschematron\init.py -> build\lib.win-amd64-cpython-311\lxml\isoschematron copying src\lxml\etree.h -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\etree_api.h -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\lxml.etree.h -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\lxml.etree_api.h -> build\lib.win-amd64-cpython-311\lxml copying src\lxml\includes\c14n.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\config.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\dtdvalid.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\etreepublic.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\htmlparser.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\relaxng.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\schematron.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\tree.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\uri.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\xinclude.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\xmlerror.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\xmlparser.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\xmlschema.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\xpath.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\xslt.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\init__.pxd -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\etree_defs.h -> build\lib.win-amd64-cpython-311\lxml\includes copying src\lxml\includes\lxml-version.h -> build\lib.win-amd64-cpython-311\lxml\includes creating build\lib.win-amd64-cpython-311\lxml\isoschematron\resources creating build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\rng copying src\lxml\isoschematron\resources\rng\iso-schematron.rng -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\rng creating build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl copying src\lxml\isoschematron\resources\xsl\RNG2Schtrn.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl copying src\lxml\isoschematron\resources\xsl\XSD2Schtrn.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl creating build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_abstract_expand.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_dsdl_include.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_schematron_message.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_schematron_skeleton_for_xslt1.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_svrl_for_xslt1.xsl -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\readme.txt -> build\lib.win-amd64-cpython-311\lxml\isoschematron\resources\xsl\iso-schematron-xslt1 running build_ext building 'lxml.etree' extension creating build\temp.win-amd64-cpython-311 creating build\temp.win-amd64-cpython-311\Release creating build\temp.win-amd64-cpython-311\Release\src creating build\temp.win-amd64-cpython-311\Release\src\lxml "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DCYTHON_CLINE_IN_TRACEBACK=0 -Isrc -Isrc\lxml\includes -IC:\Python311\include -IC:\Python311\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\cppwinrt" /Tcsrc\lxml\etree.c /Fobuild\temp.win-amd64-cpython-311\Release\src\lxml\etree.obj -w cl: 命令行 warning D9025 :正在重写“/W3”(用“/w”) etree.c C:\Users\delic\AppData\Local\Temp\pip-install-iqfkbjsa\lxml_cd5c3d18348a4806a67c6f6702409d0c\src\lxml\includes/etree_defs.h(14): fatal error C1083: 无法打开包括文件: “libxml/xmlversion.h”: No such file or directory Compile failed: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe' failed with exit code 2 creating Users creating Users\delic creating Users\delic\AppData creating Users\delic\AppData\Local creating Users\delic\AppData\Local\Temp "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -I/usr/include/libxml2 "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\cppwinrt" /TcC:\Users\delic\AppData\Local\Temp\xmlXPathInitm54nz0ww.c /FoUsers\delic\AppData\Local\Temp\xmlXPathInitm54nz0ww.obj xmlXPathInitm54nz0ww.c C:\Users\delic\AppData\Local\Temp\xmlXPathInitm54nz0ww.c(1): fatal error C1083: 无法打开包括文件: “libxml/xpath.h”: No such file or directory error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe' failed with exit code 2

  Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
  *********************************************************************************
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for lxml Running setup.py clean for lxml Failed to build lxml ERROR: Could not build wheels for lxml, which is required to install pyproject.toml-based projects

dataabc commented 1 year ago

您单独安装lxml看看。

MystFeng commented 1 year ago

您单独安装lxml看看。

已经试过了，如果pip install lxml，就会提示要求已满足在路径c:\python311\lib\site-packages(4.9.0) 然而安装依赖时setup.py还是提示没有装libxml2。我的python版本3.11.2，我该重装吗？

dataabc commented 1 year ago

您再搜索看看。我看https://github.com/Yenthe666/InstallScript/issues/243 有类似的问题，不知道是否有帮助。

Weydon-Ding commented 1 year ago

我之前在Windows平台上也报了这个错误，后来换了Mac和Linux倒是没有问题。可以试试使用docker部署，我已经pull request了Dockerfile