kennethreitz / maya

Datetimes for Humans™
MIT License
3.41k stars 197 forks source link

Maya gains an extra microsecond when parsing dates after the year 2243 #6

Closed alexwlchan closed 7 years ago

alexwlchan commented 7 years ago

Consider the following program:

import datetime
import maya

d = datetime.datetime(2243, 1, 1, 0, 0, 0, 1)
print(d)
print(str(d))

parsed_d = maya.parse(str(d)).datetime()
print(parsed_d)
print(str(parsed_d))

assert d == parsed_d
2243-01-01 00:00:00.000001
2243-01-01 00:00:00.000001
2243-01-01 00:00:00.000002+00:00
2243-01-01 00:00:00.000002+00:00
Traceback (most recent call last):
  File "counterexample.py", line 15, in <module>
    assert d == parsed_d
AssertionError

This is a bug introduced somewhere in Maya – I see that Maya is using dateutil for the main machinery of its parse function, but if I try this with dateutil everything is fine:

import datetime
from dateutil import parser

d = datetime.datetime(2243, 1, 1, 0, 0, 0, 1)
print(d)
print(str(d))

parsed_d = parser.parse(str(d))
print(parsed_d)
print(str(parsed_d))

assert d == parsed_d
2243-01-01 00:00:00.000001
2243-01-01 00:00:00.000001
2243-01-01 00:00:00.000001
2243-01-01 00:00:00.000001

Environment details:

$ pip freeze
dateparser==0.5.0
humanize==0.5.1
iso8601==0.1.11
jdatetime==1.8.1
-e git+git@github.com:kennethreitz/maya.git@6a4ddff215edbdce4a0f5b8bd5ba87710697b7f9#egg=maya-master
python-dateutil==2.6.0
pytz==2016.10
regex==2016.11.21
ruamel.yaml==0.13.4
six==1.10.0
tzlocal==1.3
umalqurra==0.2
wheel==0.24.0

I don’t actually care if you fix this (by 2243, I’ll either be dead, or hopefully have better things to do with my time), but having found it, I thought I’d share.

Yes, I know I’m a terrible person for finding this.

kennethreitz commented 7 years ago

i think one second of accuracy is acceptable for most use cases :)

alexwlchan commented 7 years ago

But think of the children*!

Seriously though, this looks pretty cool. Props for writing it. (I’ll stop trying to break it now.) 🙂


*’s children’s children’s … children’s need for second-level accurate, human-friendly datetime libraries in the future.

mithrandi commented 7 years ago

For those who care: the bug is caused by the float type having insufficient precision to represent the underlying POSIX timestamp correctly. The further away you get from the POSIX epoch (in either direction), the more precision you'll lose in the timestamp.

kennethreitz commented 7 years ago

I think this is OK.

glyph commented 7 years ago

IMHO this is a valid bug and should be addressed. Perhaps not with super high priority, but although it creeps up past a second in 2243, the inaccuracy is creeping up by various sub-second values on the way there.

The solution might not be all that hard, either. What about just storing the date internally as a Decimal?

mithrandi commented 7 years ago

@glyph The inaccuracy creeps up past a microsecond in 2243; it only creeps up past a second in the year 285572427 or so. I don't think using Decimal internally will help much, since datetime is heavily involved here, and does not support conversion from Decimal.

glyph commented 7 years ago

@mithrandi Thanks for the correction. The title of the bug did sound pretty extreme (gaining an "extra second") but time is weird and float time doubly so, so I didn't double check :).