Boris-code / feapder

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度

Other

2.88k stars 476 forks source link

已经升级最新版本

运行环境 Python 3.11.2 Playwrigh for Python 1.32.1 Pycharm 2023.1

问题 ERROR | logging:exception:line:1524 | global flags not at the start of the expression at position 2

截图 img-2023 04 10-09 07 45

代码

class AirspiderLawsMonthly(feapder.AirSpider):
    def start_requests(self):
        yield feapder.Request(url="https://www.pkulaw.com", render=True)

    def parse(self, request, response):
        # 使用playwright操作浏览器对象
        driver: PlaywrightDriver = response.driver
        page: Page = driver.page

        # 获取列表数据总数及二次检索
        laws_data_list = response.xpath("(//div[@class='grouping-title'])[1]")
        for laws_data in laws_data_list:
            grouping_title = laws_data.xpath("(//div[@class='grouping-title'])[1]").extract_first()
            print(grouping_title)

        # 提取网站title
        print(response.xpath("//title/text()").extract_first())
        # 提取网站描述
        print(response.xpath("//meta[@name='description']/@content").extract_first())
        print("网站地址: ", response.url)

Boris-code / feapder

ERROR | logging:exception:line:1524 | global flags not at the start of the expression at position 2 #210

代码