search

scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.3k stars 1.4k forks source link

How to use perform login of portia? #601

Closed Peter311 closed 7 years ago

Peter311 commented 8 years ago

When crawl one website, I need to login the website first.So I confige the perform login of portia.But it still don't work.Can you give a video address or tutorial,How to use perform login of portia?Thanks very much!

ruairif commented 7 years ago

If the login requires Javascript interaction login is not possible at the moment

taylorsmurphy commented 6 years ago

Is this still true? This issue pops up at the top of google seaches for "portia login" so it could be useful to flag it as fixed, not being developed (and so don't use portia if javascript might be involved), or whatever the case may be.

edit for posterity:

http://portia.readthedocs.io/en/latest/spiders.html?highlight=login#configuring-login-details

Configuring login details If you need to log into a site, you can configure login details by ticking ‘Perform login’ in the spider properties menu. Here you can set the login URL, username and password.

Enabling JavaScript You can enable JavaScript in your spider by ticking Enable JavaScript in the spider properties menu. Note that you’ll need to set the SPLASH_URL Scrapy setting to your Splash endpoint URL for JavaScript to work during the crawl.

So presumably you can now login and use javascript and, one would naturally assume, login when there's javascript involved