DateTime format problems with OnpointCCU

[stoklund]

I'm trying to download transactions from OnpointCCU. I'm seeing some weird problems with the formatting of DTSTART. The server is returning a HTTP 403 error if it doesn't like the timezone format:

• 19961005132200.124[0:GMT] returns HTTP 403.
• 19961005132200.124 works fine.
• 19961005132200.124[0] returns HTTP 403.
• 19961005132200.124[-6] works fine. This is also the timezone used in the server's own replies.
• 19961005132200.124[-6:CST] works too.

Clearly, this server is having some issues, and ofxtools is not to blame. It looks like the server wants timestamps in its own timezone, or none at all. (Who knows how it interprets those).

I would like to teach ofxtools to talk to this server. Do you have any advice on how to do that?

[csingley]

What exactly are you doing to generate your OFX request? Are you using the ofxget script?

Have you read the docstring at the top of the file in ofxtools/Client.py ? Have you tried that approach, just fixing the tzinfo to central?

I think you should be able to do what you require by directly using the client library. However, you probably won't be able to use the ofxtools script.

[stoklund]

I started with the ofxget script, but I kept getting the 403 error, so I switched over to my own Python script using OFXClient(...) directly. I copied parameters from fi.cfg:

client = OFXClient(
            url="https://secure.onpointcu.com/5123_DirectConnect/ofx.ashx",
            userid="...",
            org="OnPointCCU Q2",
            fid="16936",
            version=160,
            prettyprint=True,
            bankid="...",
        )

I suspected the timezone when I compared logs from AqBanking and Banktivity which are both able to talk to this server.

Looking at the function DateTime._unconvert_datetime() in Types.py, it looks to me like the timestamps are always converted to the UTC timezone when composing a request. I experimented with the timezone format by directly editing that function:

        fmt = "%Y%m%d%H%M%S.{}".format(millisec_str)
        return value.strftime(fmt)

I'll try giving my datetime objects a Central timezone instead of UTC, but this line makes me think that won't work:

        # Transform to GMT
        value = value.astimezone(utils.UTC)

While I was experimenting, I also found that Vanguard rejects a [-6] timezone, but they are happy with the [0:GMT] that ofxtools is currently generating. Weird.

[stoklund]

Okay, I verified that theory: I created datetime objects with a central timezone:

tz = pytz.timezone('US/Central')
...
start = datetime.fromtimestamp(timestamp, tz) - timedelta(days=7)

This creates a datetime which prints itself as 2021-01-25 17:03:30.174239-06:00. The corresponding OFX request contains <DTSTART>20210125230330.174[0:GMT]</DTSTART> as expected from reading the DateTime._unconvert_datetime() code.

[csingley]

Yeah I think you've got it right; now I remember forcing Zulu time.

I still think we should probably enforce UTC on the incoming type conversion, but I'm not sure it's necessary on outgoing "unconversions".

What happens if you just comment out that line?i

[Edit] You'd have to fix the fmt down a few lines. I think that was my main motivation for forcing UTC going out. This should be fixable though, if we have TZ-aware datetime objects.

[stoklund]

What happens if you just comment out that line?

You'd have to fix the fmt down a few lines. I think that was my main motivation for forcing UTC going out. This should be fixable though, if we have TZ-aware datetime objects.

Let me try changing that function to preserve whatever timezone is already on the datetime objects.

[stoklund]

Okay, #115 is a WIP PR which preserves the timezone. With US/Central datetimes, this generates [-6:CST] timezones in OFX, and these are accepted by both OnPoint and Vanguard.

I also tried with UTC datetimes which are formatted as [+0:UTC] with this patch. That format was also accepted by both OnPoint and Vanguard.

I am a little confused if these servers are rejecting the timezone or the formatting of the timezone. (Onpoint rejects [0:GMT] but accepts [+0:UTC]. Vanguard rejects [-6] but accepts [-6:CST]).

If you think this is the way to go, I can clean up the PR and add some tests.

[csingley]

Looks good! Thank you, please do.

[stoklund]

Closing since #115 was merged. Thanks!