captin411 / ofxclient

Bank transaction downloader and python OFX client libraries
MIT License
261 stars 89 forks source link

Adding Discover Card does not work #15

Open jat255 opened 9 years ago

jat255 commented 9 years ago

Hello,

Thanks for such a great program. I'm trying to add my Discover Card to the ofxclient list, but it appears to not be working. I search for 'Discover', select 'Discover Card', and then enter my user name and password. There are no errors given about any sort of authentication failure or anything, but just returns to the main menu (with no additional account added). If I enter a bogus password, it tells me the authentication has failed, so I assume it is successfully logging in somehow, but just not adding any accounts.

Looking at ofxclient.ini, nothing is added to the file when adding the account, even if I remove the initial ofxclient.ini file that was there before.

Is there anyway to go about debugging this? Looking at www.ofxhome.com, it doesn't seem like people have had any real problems lately with Discover Card...

Thanks, Josh

jat255 commented 9 years ago

I cloned the repo locally and added some debugging output. It looks like the authentication is going through, but I'm getting a response that says "I'm sorry, the system is unavailable right now, please try again later."

I was playing around with this some more, and I could get my Discover Card to work fine in KMyMoney and with ofxtools. I'm not sure what was different about these...

I tried to manually add an account to the ofxclient.ini file. I copied a working Chase credit card configuration setting and gave it a custom 64-length local_id and added in the appropriate OFX parameters for Discover Card from www.ofxhome.com (and my account number). I then added my credentials to the Windows Credential Manager using the <local_id>institution.username@ofxclient and `<local_id>institution.password@ofxclient patterns that the other institutions used. Once I had this in place, the account downloads

Here is the config section as I used it:

[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
description = Credit - Discover Card
institution.broker_id = 
institution.client_args.app_id = QWIN
institution.client_args.app_version = 2200
institution.client_args.id = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
institution.client_args.ofx_version = 102
institution.description = Discover card
institution.id = 7101
institution.org = Discover Financial Services
institution.password = %{secured}
institution.url = https://ofx.discovercard.com
institution.username = %{secured}
local_id = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
number = XXXXXXXXXXXXXXXX

I think what might be going on is that the account lookup is not getting the credit card account number, so then when it makes the second request to actually download the transactions, nothing happens. I'm not sure how to fix that though.

nhinds commented 8 years ago

I know this is an old issue, but for others who encounter the same thing... I ran into the same error message using Discover's OFX feed with another client. The problem seems to be that Discover doesn't like its OFX endpoints hit very quickly. An account info request (ACCTINFOTRNRQ) immediately followed by a statement request (CREDITCARDMSGSRQV1) failed on the second request. Waiting 5 seconds between the two requests made everything work fine for me.

(I know, adding waits to code is never the right solution, but it's at least a workaround)

captin411 commented 8 years ago

Is there any particular OFX error code that you noticed like "please slow down" that we could programmatically determine to throttle things?

David Bartle

On May 14, 2016, at 7:19 PM, Nicholas Hinds notifications@github.com wrote:

I know this is an old issue, but for others who encounter the same thing... I ran into the same error message using Discover's OFX feed with another client. The problem seems to be that Discover doesn't like its OFX endpoints hit very quickly. An account info request (ACCTINFOTRNRQ) immediately followed by a statement request (CREDITCARDMSGSRQV1) failed on the second request. Waiting 5 seconds between the two requests made everything work fine for me.

(I know, adding waits to code is never the right solution, but it's at least a workaround)

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

nhinds commented 8 years ago

Sadly not. When sending two identical OFX requests in quick succession, the second response just looks like:

<OFX>
  <SIGNONMSGSRSV1>
    <SONRS>
      <STATUS>
        <CODE>2000
          <SEVERITY>ERROR
            <MESSAGE>I'm sorry, the system is unavailable right now, please try again later.
      </STATUS>
      <DTSERVER>20160517234227.157[-4:EDT]
      <LANGUAGE>ENG
      <FI>
        <ORG>Discover Financial Services
        <FID>7101
      </FI>
    </SONRS>
  </SIGNONMSGSRSV1>
  <CREDITCARDMSGSRSV1>
    <CCSTMTENDTRNRS>
      <TRNUID>uuid-sent-by-client
      <STATUS>
        <CODE>15500
        <SEVERITY>ERROR
      </STATUS>
    </CCSTMTENDTRNRS>
  </CREDITCARDMSGSRSV1>
</OFX>

From what I understand of the spec, error 2000 is "General error" and error 15500 is "Signon invalid", neither of which are anything to do with the real reason for the failure (rate limiting)

A quick check (using sleep and curl) seems to indicate that a limit of 1s consistently fails with an error similar to the above, 1.5s fails sometimes and succeeds sometimes, and 2 seconds always works. Not super helpful for determining whether we're being throttled, though.

jantman commented 7 years ago

This should have been fixed in #37

JohnStarich commented 5 years ago

I'm still seeing this issue with my Discover Card, with an identical response to @nhinds. Here's the response headers section from the verbose output:

DEBUG:root:---- response ----
DEBUG:root:{'fp': None, 'debuglevel': 0, '_method': 'POST', 'headers': <http.client.HTTPMessage object at 0x7f4815b3d2b0>,
    'msg': <http.client.HTTPMessage object at 0x7f4815b3d2b0>, 'version': 11, 'status': 200,
    'reason': 'OK', 'chunked': False, 'chunk_left': 'UNKNOWN', 'length': 0, 'will_close': False, 'code': 200}
DEBUG:root:Headers: [('Content-Type', 'application/x-ofx'), ('Date', 'Tue, 16 Apr 2019 01:54:21 GMT'),
    ('Content-Length', '610'), ('Connection', 'keep-alive'), ('Set-Cookie', 'TS01ba2681=<redacted>; Path=/; HTTPOnly'),
    ('Set-Cookie', 'DCID=www21; path=/; domain=ofx.discovercard.com')]

Based on the message, perhaps we could detect "unavailable right now" with a 200 status code and then retry once with a delay (for Discover)?

Could also get fancy and use a rate limiter on the def _message function or something 🤷‍♂️

chaserhkj commented 5 years ago

Confirm this bug still persists, I could not add my Discover card and it's showing the same error.

Probably we still need a rate limiter besides #37