lixi5338619 / asyncpy

使用asyncio和aiohttp开发的轻量级异步协程web爬虫框架
110 stars 27 forks source link

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001863E6E2790> #7

Open yuzhiyizhan opened 1 year ago

yuzhiyizhan commented 1 year ago

image pixiy.py

# -*- coding: utf-8 -*-
from middlewares import middleware
from asyncpy.spider import Spider
from loguru import logger

class PixivSpider(Spider):
    name = 'pixiv'

    start_urls = []
    cookies = {
        ***
    }
    headers = {
       ***
    }

    async def start_requests(self):
        user_id = list(
            set([12651854, 13926781]))

        for i in user_id:
            url = f'https://www.pixiv.net/ajax/user/{i}/profile/all?lang=zh'
            yield self.request(url=url, callback=self.parse, cookies=self.cookies, meta={'user_id': i}, method="GET",
                               headers=self.headers)
            break

    async def parse(self, response):
        logger.debug(response.text)

PixivSpider.start(middleware=middleware)

大佬你好,我编写上面程序时会报 RuntimeError: Event loop is closed 我的python版本3.8 asyncpy 1.2.0 aiohttp 3.8.1

我不是很会python的异步编程,请大佬指点

tuhaolam commented 1 year ago

为什么要在yield后面加break,这样就只会执行一次就退出了

yuzhiyizhan commented 1 year ago

我在测试的时候加的,删掉break就行

---- 回复的原邮件 ---- | 发件人 | @.> | | 日期 | 2023年05月17日 02:06 | | 收件人 | @.> | | 抄送至 | @.>@.> | | 主题 | Re: [lixi5338619/asyncpy] Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000001863E6E2790> (Issue #7) |

为什么要在yield后面加break,这样就只会执行一次就退出了

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>