search

nicksergeant / billmonster

A suite of Selenium scripts to automate the retrieval of current balances due for several financial providers.
MIT License
7 stars 3 forks source link

Capital One provider does not work anymore #2

Open certik opened 9 years ago

certik commented 9 years ago

It raises the following exception:

$ python billmonster.py

Capital One (harry)
Traceback (most recent call last):
  File "billmonster.py", line 66, in <module>
    main()
  File "billmonster.py", line 45, in main
    PROVIDERS[provider](user, False, browser, index)
  File "/home/ondrej/repos/billmonster/capitalone.py", line 49, in capitalone
    account_type = b.find_element_by_css_selector('option[value="credit cards"]')
  File "/home/ondrej/repos/hashstack/default/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 374, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "/home/ondrej/repos/hashstack/default/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
    {'using': by, 'value': value})['value']
  File "/home/ondrej/repos/hashstack/default/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
    self.error_handler.check_response(response)
  File "/home/ondrej/repos/hashstack/default/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: {"method":"css selector","selector":"option[value=\"credit cards\"]"}
Stacktrace:
    at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpdm14mV/extensions/fxdriver@googlecode.com/components/driver-component.js:9641:26)
    at FirefoxDriver.prototype.findElement (file:///tmp/tmpdm14mV/extensions/fxdriver@googlecode.com/components/driver-component.js:9650:3)
    at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmpdm14mV/extensions/fxdriver@googlecode.com/components/command-processor.js:11635:16)
    at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmpdm14mV/extensions/fxdriver@googlecode.com/components/command-processor.js:11640:7)
    at DelayedCommand.prototype.execute/< (file:///tmp/tmpdm14mV/extensions/fxdriver@googlecode.com/components/command-processor.js:11582:5)
nicksergeant commented 9 years ago

Yeah, I gave up on this project a swift week or so after I originally wrote it. Turns out these providers change their HTML extremely frequently.

certik commented 9 years ago

Are you aware of any other way to get the data from these providers in a programmatic way? I wonder how mint.com is mining these providers. Unfortunately, Mint also doesn't seem to have an API, but somebody wrote a screen scraping API for Mint here: https://github.com/mrooney/mintapi.

nicksergeant commented 9 years ago

Mint uses http://www.yodlee.com/ for collecting data from your bank account - though my guess is that it's 1) expensive and 2) not very open.

certik commented 9 years ago

Yeah, the only cheap way is to scrape it ourselves. Here is how Wesabe did it: https://github.com/wesabe/ssu/blob/563cc4f89e058e28fc607f33c5fed5dc548b2b05/application/chrome/content/wesabe/fi-scripts/com/capitalone/credit.js