revisit time handling to ensure UTC everywhere

[rouge8]

Timezone dependent test failures; recheck all time handling code to ensure all timestamps are forced to utc or to naive with implied UTC.

Two of the ipfix.message tests fail when I run them against master:

$ python test.py
**********************************************************************
File "/Users/rouge8/forks/python-ipfix/ipfix/message.py", line 140, in ipfix.message
Failed example:
    for rec in msg.namedict_iterator():
       print(sorted(rec.items()))
Expected:
    [('destinationIPv4Address', IPv4Address('10.5.6.7')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 12, 0, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 27), ('sourceIPv4Address', IPv4Address('10.1.2.3'))]
    [('destinationIPv4Address', IPv4Address('10.12.13.14')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 12, 0, 2, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 33), ('sourceIPv4Address', IPv4Address('10.8.9.11'))]
    [('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 12, 0, 4, tzinfo=datetime.timezone.utc)), ('myNewInformationElement', "Grüezi, Y'all")]
Got:
    [('destinationIPv4Address', IPv4Address('10.5.6.7')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 18, 0, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 27), ('sourceIPv4Address', IPv4Address('10.1.2.3'))]
    [('destinationIPv4Address', IPv4Address('10.12.13.14')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 18, 0, 2, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 33), ('sourceIPv4Address', IPv4Address('10.8.9.11'))]
    [('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 18, 0, 4, tzinfo=datetime.timezone.utc)), ('myNewInformationElement', "Grüezi, Y'all")]
**********************************************************************
File "/Users/rouge8/forks/python-ipfix/ipfix/message.py", line 155, in ipfix.message
Failed example:
    for rec in msg.tuple_iterator(ielist):
        print(rec)
Expected:
    (datetime.datetime(2013, 6, 21, 12, 0, tzinfo=datetime.timezone.utc), 27)
    (datetime.datetime(2013, 6, 21, 12, 0, 2, tzinfo=datetime.timezone.utc), 33)
Got:
    (datetime.datetime(2013, 6, 21, 18, 0, tzinfo=datetime.timezone.utc), 27)
    (datetime.datetime(2013, 6, 21, 18, 0, 2, tzinfo=datetime.timezone.utc), 33)
**********************************************************************
1 items had failures:
   2 of  40 in ipfix.message
***Test Failed*** 2 failures.

[britram]

Let me guess, you're East Coast.

Python's time handling is deeply, deeply confused (a symptom of an overly complicated design) and I've still not figured out how to reliably force it to do everything in UTC, always. I was pretty sure the code in ipfix.types handling time encode/decode worked, but apparently failed to test on a machine not in UTC+2.

Will modify this issue to reflect the problem here.

Andy Freeland wrote:

Two of the |ipfix.message| tests fail when I run them against master:

|$ python test.py

---

File "/Users/rouge8/forks/python-ipfix/ipfix/message.py", line 140, in ipfix.message Failed example: for rec in msg.namedict_iterator(): print(sorted(rec.items())) Expected: [('destinationIPv4Address', IPv4Address('10.5.6.7')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 12, 0, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 27), ('sourceIPv4Address', IPv4Address('10.1.2.3'))] [('destinationIPv4Address', IPv4Address('10.12.13.14')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 12, 0, 2, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 33), ('sourceIPv4Address', IPv4Address('10.8.9.11'))] [('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 12, 0, 4, tzinfo=datetime.timezone.utc)), ('myNewInformationElement', "Grüezi, Y'all")] Got: [('destinationIPv4Address', IPv4Address('10.5.6.7')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 18, 0, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 27), ('sourceIPv4Address', IPv4Address('10.1.2.3'))] [('destinationIPv4Address', IPv4Address('10.12.13.14')), ('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 18, 0, 2, tzinfo=datetime.timezone.utc)), ('packetDeltaCount', 33), ('sourceIPv4Address', IPv4Address('10.8.9.11'))] [('flowStartMilliseconds', datetime.datetime(2013, 6, 21, 18, 0, 4, tzinfo=datetime.timezone.utc)), ('myNewInformationElement', "Grüezi, Y'all")]

---

File "/Users/rouge8/forks/python-ipfix/ipfix/message.py", line 155, in ipfix.message Failed example: for rec in msg.tuple_iterator(ielist): print(rec) Expected: (datetime.datetime(2013, 6, 21, 12, 0, tzinfo=datetime.timezone.utc), 27) (datetime.datetime(2013, 6, 21, 12, 0, 2, tzinfo=datetime.timezone.utc), 33) Got: (datetime.datetime(2013, 6, 21, 18, 0, tzinfo=datetime.timezone.utc), 27) (datetime.datetime(2013, 6, 21, 18, 0, 2, tzinfo=datetime.timezone.utc), 33)

---

1 items had failures: 2 of 40 in ipfix.message _Test Failed_ 2 failures. |

— Reply to this email directly or view it on GitHub https://github.com/britram/python-ipfix/issues/7.

[britram]

Working on this in develop

[britram]

Well that was quick. Fixed in develop branch.