search

Ehsan-U / scrapy-nodriver

Nodriver integration for Scrapy
10 stars 0 forks source link

Provide a way to set browser arguments. #4

Open paxapy opened 5 days ago

paxapy commented 5 days ago

For my project I got 403 right away from the site i'm trying to access, so as a first step I decided to set NODRIVER_HEADLESS = False and got the following error:

raise Exception(
celery-1    | Exception: 
celery-1    |                 ---------------------
celery-1    |                 Failed to connect to browser
celery-1    |                 ---------------------
celery-1    |                 One of the causes could be when you are running as root.
celery-1    |                 In that case you need to pass no_sandbox=True

Yet I have some experience with scrapy-playwright which I want to replace, I know that there is a corresponding setting called PLAYWRIGHT_LAUNCH_OPTIONS, and it seems that nodriver itself also allowing to provide these arguments.

Don't you think it will be convenient to have the ability to set starting options from settings? I think I can even create a PR for this as I'm going to do it anyways.

Ehsan-U commented 5 days ago

@paxapy feel free to go ahead & create PR