search

xianhu / PSpider

简单易用的Python爬虫框架，QQ交流群：597510560
https://github.com/xianhu/PSpider
BSD 2-Clause "Simplified" License
1.83k stars 504 forks source link

i Cant run the Tool #32

Closed Tiger-Hacker closed 2 years ago

Tiger-Hacker commented 4 years ago

Traceback (most recent call last): File "test.py", line 13, in from bs4 import BeautifulSoup ModuleNotFoundError: No module named 'bs4'

zjlx commented 4 years ago

pip install beautifulsoup4

Tiger-Hacker commented 4 years ago

On Fri, May 15, 2020, 11:27 PM zjlx notifications@github.com wrote:

2020-05-16 00:09:11,409 WARNING ThreadPool has been initialized

2020-05-16 00:09:11,411 WARNING ThreadPool starts working: urls_count=1, fetchers_num=10

Traceback (most recent call last):

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/synchronize.py", line 28, in 

from _multiprocessing import SemLock, sem_unlink

ImportError: cannot import name 'SemLock' from 'multiprocessing' (/data/data/com.termux/files/usr/lib/python3.8/lib-dynload/ multiprocessing.cpython-38.so)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "test.py", line 126, in 

test_spider()

File "test.py", line 117, in test_spider

web_spider.start_working(fetchers_num=10)

File "/data/data/com.termux/files/home/website/PSpider/spider/concurrent/threads_pool.py", line 105, in start_working

self._thread_parser = ParseThread("parser", self._inst_parser, self)

if self._inst_parser else None

File "/data/data/com.termux/files/home/website/PSpider/spider/concurrent/threads_inst/parse.py", line 23, in init

self._pool_multiprocssing = multiprocessing.Pool()

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/context.py", line 119, in Pool

return Pool(processes, initializer, initargs, maxtasksperchild,

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/pool.py", line 191, in init

self._setup_queues()

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/pool.py", line 343, in _setup_queues

self._inqueue = self._ctx.SimpleQueue()

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/context.py", line 113, in SimpleQueue

return SimpleQueue(ctx=self.get_context())

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/queues.py", line 336, in init

self._rlock = ctx.Lock()

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/context.py", line 67, in Lock

from .synchronize import Lock

File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/synchronize.py", line 30, in 

raise ImportError("This platform lacks a functioning sem_open" +

ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.

this is what happens when i run the toop after installing it

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/xianhu/PSpider/issues/32#issuecomment-629398065, or unsubscribe https://github.com/notifications/unsubscribe-auth/APRROU4XMI23JJL6QITY3WLRRV665ANCNFSM4NBIF77Q .

zjlx commented 4 years ago

I think it may be not support python 3.8, can you try python 3.6/3.7?

Tiger-Hacker commented 4 years ago

How?

On Sat, May 16, 2020, 7:22 AM zjlx notifications@github.com wrote:

I think it may be not support python 3.8, can you try python 3.6/3.7?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/xianhu/PSpider/issues/32#issuecomment-629571576, or unsubscribe https://github.com/notifications/unsubscribe-auth/APRROU5HVQTP3PWPJSJWRJDRRXWXPANCNFSM4NBIF77Q .