howie6879 / ruia

Async Python 3.6+ web scraping micro-framework based on asyncio
https://www.howie6879.com/ruia/
Apache License 2.0
1.75k stars 181 forks source link

运行示例代码报错 #138

Closed qgyhd1234 closed 3 years ago

qgyhd1234 commented 3 years ago

我参考的是 https://github.com/howie6879/ruia/blob/master/docs/en/tutorials/item.md 里的代码

import asyncio
from ruia import Item, TextField, AttrField

class PythonDocumentationItem(Item):
    title = TextField(css_select='title')
    tutorial_link = AttrField(xpath_select="//a[text()='Tutorial']", attr='href')

async def main():
    url = 'https://docs.python.org/3/'
    item = await PythonDocumentationItem.get_item(url=url)
    print(item.title)
    print(item.tutorial_link)

if __name__ == '__main__':
    # Python 3.7 required
    asyncio.run(main())

运行能获取到正常的结果 3.9.5 Documentation tutorial/index.html 但是会报错,提示RuntimeError: Event loop is closed 完整的运行结果如下所示:

[2021:05:06 14:11:55] INFO  Request <GET: https://docs.python.org/3/>
3.9.5 Documentation
tutorial/index.html
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000018664A679D0>
Traceback (most recent call last):
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python39\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python39\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

PC :win10 64bit Python :3.9.4 64bit

howie6879 commented 3 years ago

请问你使用的ruia版本是?


In [1]: import ruia

In [2]: ruia.__version__
Out[2]: '0.8.3'
qgyhd1234 commented 3 years ago

0.8.3 image

howie6879 commented 3 years ago

你好,我这边手头上没有windows,麻烦你这边测一下看看,直接在编辑器跳到源码的spider.py文件,在前面加上

import asyncio

if sys.version_info[0] == 3 and sys.version_info[1] >= 8 and sys.platform.startswith('win'):
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

看是否解决你的问题?

qgyhd1234 commented 3 years ago

你好,我这边手头上没有windows,麻烦你这边测一下看看,直接在编辑器跳到源码的spider.py文件,在前面加上

import asyncio

if sys.version_info[0] == 3 and sys.version_info[1] >= 8 and sys.platform.startswith('win'):
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

看是否解决你的问题?

我在家里测试,python 是 3.8.3 64 bit,win,也会提示RuntimeError: Event loop is closed,按照你的说明修改后,解决了问题。3.9 的 python 明天去公司测试下。

howie6879 commented 3 years ago

好的 麻烦测试下

qgyhd1234 commented 3 years ago

测试没有问题了

qgyhd1234 commented 3 years ago

顺便问下有没有交流 ruia 的 qq/微信群啊?

howie6879 commented 3 years ago

https://gitee.com/howie6879/oss/raw/master/uPic/qrcode_for_gh_3f02ace79dfb_258.jpg

我公众号右下角有我微信,我拉你

howie6879 commented 3 years ago

已经更新ruia到v0.8.4

qgyhd1234 commented 3 years ago

已解决,谢谢作者