search

scrapy / loginform

Fill HTML login forms automatically
270 stars 79 forks source link

Not all form fields are included in the posting #1

Open kalessin opened 12 years ago

kalessin commented 12 years ago

In this login page, for example,

https://www.beyondtherack.com/auth/login

there is the submit button element with name _submit inside the login form, which is not being included in the posting, but it is required for the login to be successful (tested)

The problem is that the lxml method form.form_values() alone does not include all the involved fields in the form. Seems it only includes the input elements.

pablohoffman commented 12 years ago

@kalessin check the code on scrapy.http.request.form, it has support for submitting values of clickable elements (which works like that by default unless dont_click is passed). You may be able to get some ideas from there to submit a PR to fix this.

There's a lot of code in that module though, probably much more than we need here, so only import strictly what you need.