scrapy-plugins / scrapy-playwright

🎭 Playwright integration for Scrapy
BSD 3-Clause "New" or "Revised" License
992 stars 108 forks source link

How to use playwright-stealth ? #245

Closed josearodrigueze closed 10 months ago

elacuesta commented 10 months ago

Assuming you're referring to https://pypi.org/project/playwright-stealth/, it should be possible to do it with https://github.com/scrapy-plugins/scrapy-playwright#playwright_page_init_callback. I haven't tested it myself, but something like this:

from playwright_stealth import stealth_async

async def init_page(page, request):
    await stealth_async(page)

class StealthSpider(scrapy.Spider):
    name = "stealth"

    def start_requests(self):
        yield scrapy.Request(
            url="https://example.org",
            meta={
                "playwright": True,
                "playwright_page_init_callback": init_page,
            },
        )
josearodrigueze commented 10 months ago

It works perfect thanks for the help. I really appreciate your help, have a great day.