unitedstates / contact-congress

Sending electronic written messages to members of Congress by reverse engineering their contact forms.
https://theunitedstates.io/contact-congress/
Creative Commons Zero v1.0 Universal
633 stars 212 forks source link

[U000031] Rep. Fred Upton #133

Closed konklone closed 10 years ago

konklone commented 11 years ago

Rep. Fred Upton: http://www.house.gov/upton

Likely contact form URL to start with: http://www.house.gov/writerep

j-ro commented 10 years ago

grabbing this one

j-ro commented 10 years ago

unsure why this is failing, it's a two step one though, so the YAML was merged. file:

https://github.com/unitedstates/contact-congress/blob/master/members/U000031.yaml

sinak commented 10 years ago

So making it click thrice seems to have made it work some percentage of the time. Really weird one. @Hainish any thoughts?

sinak commented 10 years ago

Yep, so in my browser it requires clicking twice, but for congress-forms-test it seems to require a variable number, hence it failing. I'm not really sure what to do here.

j-ro commented 10 years ago

Yeah, I have to click twice too. But it still fails a lot in the tester...

sinak commented 10 years ago

Yeah. Something like a "keep clicking submit every few seconds until success happens" flag might be one way of handling this? Don't really have any suggestions otherwise.

j-ro commented 10 years ago

@Hainish don't think so, no -- it originally had one click, @sinak added a second, but it was still intermittent. It's some javascript error and an XHR call that does something or other on the page. Maybe it has to be "click every few seconds until it works"...

Hainish commented 10 years ago

Yeah, very strange behavior. I think we'll need to add an extra instruction for "wait a certain interval," maybe half a second. This will probably come in handy for other legislators as we move forward as well.

contactcongress commented 10 years ago

Hey guys, I haven't looked through the source yet, but could it be that some validation happens on the first click and there's a dom change that results? Maybe a click-find-click approach would do it?— Dan Drinkard

On Sat, Apr 12, 2014 at 3:44 AM, Bill Budington notifications@github.com wrote:

Yeah, very strange behavior. I think we'll need to add an extra instruction for "wait a certain interval," maybe half a second. This will probably come in handy for other legislators as we move forward as well.

Reply to this email directly or view it on GitHub: https://github.com/unitedstates/contact-congress/issues/133#issuecomment-40274297

j-ro commented 10 years ago

So, we tried to clicks in a row, but it was still inconsistent. It does fire a validation function on click which may have a small DOM change, but there are also JS errors. For example, both Sina and I have to click the button twice when we use the form manually.

I think @Hainish's idea might work, though I also think maybe we want a structure that's more click - wait - scan for success - repeat (a certain number of times) in this case.

sinak commented 10 years ago

We should take a gander and see if there's any kind of DOM change, as that'd really be ideal. It looked like it was just a javascript error though.

On Sat, Apr 12, 2014 at 7:10 AM, Jason Rosenbaum notifications@github.comwrote:

So, we tried to clicks in a row, but it was still inconsistent. It does fire a validation function on click which may have a small DOM change, but there are also JS errors. For example, both Sina and I have to click the button twice when we use the form manually.

I think @Hainish https://github.com/Hainish's idea might work, though I also think maybe we want a structure that's more click - wait - scan for success - repeat (a certain number of times) in this case.

Reply to this email directly or view it on GitHubhttps://github.com/unitedstates/contact-congress/issues/133#issuecomment-40281284 .

drinks commented 10 years ago

Yeah I mostly brainlessly cribbed my command spec off of capybara because it's proven that it can drive phantomjs trivially and I didn't want to run into any surprises in implementation. Adding sleep would be the first time straying from that afaik, and I'd prefer not to do it if it's at all avoidable.

j-ro commented 10 years ago

Maybe just adding the click submit many times (say, 10?) would give us the delay we need...there's a XHR call in there, so I think that's why it sometimes fails, depending on the speed of that call.

drinks commented 10 years ago

Hm, some sort of preflight validation, then? We will probably need to work out a specific way to deal with this, as it only affects js-enabled clients. I wonder what will happen with a plain mechanize submit...— Dan Drinkard

On Sat, Apr 12, 2014 at 4:22 PM, Jason Rosenbaum notifications@github.com wrote:

Maybe just adding the click submit many times (say, 10?) would give us the delay we need...there's a XHR call in there, so I think that's why it sometimes fails, depending on the speed of that call.

Reply to this email directly or view it on GitHub: https://github.com/unitedstates/contact-congress/issues/133#issuecomment-40290781

j-ro commented 10 years ago

Yeah, I think that's it. There are a few like that (like ones that validate address really strictly, as I've noted in other tickets) but I don't think they have this specific issue.

Hainish commented 10 years ago

Works now, since there was a DOM change to look for. Next time we might not be so lucky though. If it was because of some internal javascript state, we may have had to stray from the capybara standard API and either issued a "wait" or perhaps another command, like "run_javascript"