far-flung RFC822 dates don't throw OverflowError on x64

[GoogleCodeExporter]

This test fails for me both in 5.1 and latest svn. Here are some example 
failures:
python2.4,2.5:

======================================================================
FAIL: test__parse_date_rfc822_01 (__main__.TestDateParsers)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "feedparsertest.py", line 504, in <lambda>
    return lambda self: self._check_date(f, s, t)
  File "feedparsertest.py", line 424, in _check_date
    self.assertEqual(tup, dttuple)
AssertionError: (10000, 1, 5, 4, 38, 59, 2, 5, 0) != None

----------------------------------------------------------------------
Ran 4091 tests in 9.254s

python2.6,2.7,3.1,3.2:
======================================================================
FAIL: test__parse_date_rfc822_01 (__main__.TestDateParsers)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "feedparsertest.py", line 504, in <lambda>
    return lambda self: self._check_date(f, s, t)
  File "feedparsertest.py", line 424, in _check_date
    self.assertEqual(tup, dttuple)
AssertionError: time.struct_time(tm_year=10000, tm_mon=1, tm_mday=5, tm_hour=4, 
tm_min=38, tm_sec=59, tm_wday=2, tm_yday=5, tm_isdst=0) != None

----------------------------------------------------------------------

Original issue reported on code.google.com by jo...@bergstroem.nu on 8 Dec 2011 at 2:53

[GoogleCodeExporter]

Um...I've only encountered this problem with Jython, so I've known that these 
tests would fail on a different Python implementation. However, I test on 
Python 2.4 through 3.2 and these tests all pass, so I'm curious what platform 
you're on.

Do you have a modified rfc822 module by chance? Mine throws an OverflowError 
when I run the following code:

import feedparser as fp
fp._parse_date_rfc822('Sun, 31 Dec 9999 23:59:59 -9999')

Original comment by kurtmckee on 8 Dec 2011 at 4:09

[GoogleCodeExporter]

Unfortunately not:
(Left: my install, Right: from tarball)
dev /home/jbergstroem # diff -u /usr/lib64/python2.4/rfc822.py 
Python-2.4.6/Lib/rfc822.py 
dev /home/jbergstroem # 

dev /home/jbergstroem # diff -u /usr/lib64/python3.2/email/_parseaddr.py 
Python-3.2.2/Lib/email/_parseaddr.py 
dev /home/jbergstroem # 

Are you perhaps using a twinked one? 

Original comment by jo...@bergstroem.nu on 8 Dec 2011 at 4:27

[GoogleCodeExporter]

Ha ha! No, mine were all installed from source on an x86 machine, but perhaps 
the difference is the x86 versus x64 architectures (and Jython circumvents the 
issue)? I'll modify the tests to check for both cases, but it will be after I 
check in a fix for issue 304.

Original comment by kurtmckee on 8 Dec 2011 at 4:38

• Changed state: Accepted

[GoogleCodeExporter]

Non pun intended. Different linux discs usually solves problems in different 
ways. I'll investigate as well and return.

Original comment by jo...@bergstroem.nu on 8 Dec 2011 at 4:40

[GoogleCodeExporter]

Just wanted to report back in and state that i get the same result on python2.6 
and python2.7 on OS X Lion, but not in python2.5 where tests pass. Perhaps 
skipped (number of tests executed differ)?

Original comment by jo...@bergstroem.nu on 8 Dec 2011 at 10:06

[GoogleCodeExporter]

First of all, the tests are not skipped on Lion.

Upon further investigation, these tests actually pass (only checking on latest 
svn, r674, at the moment) on a gentoo 32-bit environment – with python 2.7.2 
– while failing on the exact same version at a 64-bit one.

32-bit:
# python2.7
Python 2.7.2 (default, Oct 26 2011, 15:29:33) 
[GCC 4.5.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import feedparser
>>> feedparser._parse_date_rfc822(u'Sun, 31 Dec 9999 23:59:59 -9999')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "feedparser.py", line 3429, in _parse_date_rfc822
    return time.gmtime(rfc822.mktime_tz(tm))
  File "/usr/lib/python2.7/rfc822.py", line 949, in mktime_tz
    t = time.mktime(data[:8] + (0,))
OverflowError: mktime argument out of range

64-bit:
# python2.7
Python 2.7.2 (default, Sep 17 2011, 15:29:31) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import feedparser
>>> feedparser._parse_date_rfc822(u'Sun, 31 Dec 9999 23:59:59 -9999')
time.struct_time(tm_year=10000, tm_mon=1, tm_mday=5, tm_hour=4, tm_min=38, 
tm_sec=59, tm_wday=2, tm_yday=5, tm_isdst=0)

I know it sounds strange, but thats about the only difference I can see between 
those two systems.

Original comment by jo...@bergstroem.nu on 10 Dec 2011 at 2:22

[GoogleCodeExporter]

That doesn't sound too strange, actually: the datetime module documentation 
seems to suggest that Python relies on the system's C library for some date and 
time related functionality, so maybe that accounts for the difference. I'll 
just have to update the tests so that they don't fail if the 
platform/implementation can handle the dates.

By the way, the date parsing tests always run, but the test count differs from 
platform to platform based on a number of factors. For instance, if the utf_32 
codec isn't available (which is the case in Python 2.4 and 2.5), there are some 
encoding tests that won't run. If BeautifulSoup isn't available, the 
microformat tests won't run. There are a few other examples, and I'm 
considering adding some output that lists what tests are being skipped, but 
that's currently not high on my priorities.

Thanks for your diligence, I'll get this fixed as soon as I have an opportunity!

Original comment by kurtmckee on 10 Dec 2011 at 4:50

[GoogleCodeExporter]

I just verified this on OS X Lion, and it surely seems that 2.5 on Lion is 
32-bit and 2.6,2.7 is 64-bit. Next step would most likely be finding an 
upstream python bug, no?

Original comment by jo...@bergstroem.nu on 10 Dec 2011 at 4:55

[GoogleCodeExporter]

On the feedparser side of things it's clear that some platforms and 
implementations can handle such a far-flung date instead of throwing an 
exception, so I need to change the unit tests. I doubt that this is a Python 
bug, but rather a limitation in the system libraries that Python's using. You 
can pursue the issue into the Python standard library, but I'm already 
convinced that I need to change the unit tests.

Original comment by kurtmckee on 10 Dec 2011 at 5:41

[GoogleCodeExporter]

Fixed in r687. Thanks for reporting this!

Original comment by kurtmckee on 17 Feb 2012 at 1:29

• Changed title: far-flung RFC822 dates don't throw OverflowError on x64
• Changed state: Fixed