Doesn't work with Python 2.4

[GoogleCodeExporter]

This is documented (in the readme), but it still sucks.

And it's typically only a small amount of code that goes wrong.

Things that I know go wrong in Python 2.4 :
fetching from a tarfile will not work.  This affect preflight.py.  If all of 
the "small" inputs are already 
present in the input/ directory, then the tarfile stuff isn't needed, and the 
rest of ccc-gistemp 
should work with Python 2.4

Original issue reported on code.google.com by d...@ravenbrook.com on 11 Jan 2010 at 7:13

[GoogleCodeExporter]

Here's an example of it going wrong:

--
$ python2.4 code/fetch.py v2.inv
Extracting members from 
http://data.giss.nasa.gov/gistemp/sources/GISTEMP_sources.tar.gz ...
Traceback (most recent call last):
  File "code/fetch.py", line 273, in ?
    sys.exit(main())
  File "code/fetch.py", line 265, in main
    fetch(args)
  File "code/fetch.py", line 166, in fetch
    handler[hname](group, prefix, output)
  File "code/fetch.py", line 201, in fetch_tar
    fetch_from_tar(u, members, prefix, output)
  File "code/fetch.py", line 218, in fetch_from_tar
    tar = tarfile.open(mode='r|*', fileobj=inp)
  File "/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/tarfile.py", line 912, in open
    _Stream(name, filemode, comptype, fileobj, bufsize))
  File "/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/tarfile.py", line 836, in __init__
    self.firstmember = self.next()
  File "/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/tarfile.py", line 1588, in next
    self.fileobj.seek(self.offset)
  File "/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/tarfile.py", line 418, in seek
    self.read(remainder)
  File "/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/tarfile.py", line 437, in read
    buf = self._read(size)
  File "/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/tarfile.py", line 447, in _read
    c = len(self.dbuf)
AttributeError: _Stream instance has no attribute 'dbuf'
--

You can workaround that by saving the tarball to a real file first and opening 
that.  But it still doesn't work.  
And nor does ungzipping the tarball into a tarfile.

So.

Bottom line: Python 2.4: tarfile support is No Bloody Good.

Original comment by d...@ravenbrook.com on 11 Jan 2010 at 9:52

[GoogleCodeExporter]

There are actually two problems.

1. The Python2.4 tarfile module does not support the 'r|*' form of mode in the 
open()
function. You have to explicitly do, for example, 'r|gz' for a gzipped file.
Intelligent use of the tarfile's extension should be good enough to fix this for
Python2.4.

2. Unfortunately, the GISTEMP_sources.tar.gz file has been created by a buggy
program, which is putting leading NUL characters at the start of the 'mtime' 
field in
some file information headers. This breaks in python2.4, but python2.6 handles 
this
particular bug (possibly more by accident than design).

I can see no way to work around this in fetch.py, other than by some unpleasant
monkey patching or adding a 'fixed' tarfile module to CCCs code. The file 
itself can
be fixed by using GNUtar to unpack and then rebuild the file. (GNUtar knows how 
to
handle a lot of tarfile problems.)

Original comment by haroldsh...@gmail.com on 13 Jan 2010 at 6:57

[GoogleCodeExporter]

@paul: That's excellent detective work, thanks.  I wonder if we could rip 
tarfile.py out of Python 2.6 and drag it 
into Python 2.4 (I note it seems to be in Python, not C).

Not a priority for now, but if anyone is feeling itchy about it... go ahead.

Original comment by d...@ravenbrook.com on 13 Jan 2010 at 7:40

[GoogleCodeExporter]

@david: I'll add a suitably fixed tarfile.py into the CCC codebase.

Original comment by haroldsh...@gmail.com on 17 Jan 2010 at 12:16

[GoogleCodeExporter]

Commit 196 has changes to fix this problem.

Original comment by haroldsh...@gmail.com on 17 Jan 2010 at 1:13

[GoogleCodeExporter]

If you say r196 instead of commit 196 we get a link to the change.

Original comment by d...@ravenbrook.com on 25 Jan 2010 at 11:21

[GoogleCodeExporter]

I've tested this against my 2.4 and can attest that it works.

Original comment by nick.bar...@gmail.com on 26 Jan 2010 at 10:45

• Changed state: Fixed

[GoogleCodeExporter]

Broken again (at r261):

$ python2.4 tool/run.py
Traceback (most recent call last):
  File "tool/run.py", line 27, in ?
    import tool.step0
  File "/Users/drj/ccc-gistemp/tool/step0.py", line 17
    class Struct():
                 ^
SyntaxError: invalid syntax

Original comment by d...@ravenbrook.com on 10 Feb 2010 at 10:14

• Changed state: Accepted

[GoogleCodeExporter]

Looks like David has fixed this in revision 268, by removing the empty 
parentheses in
line 17.

Original comment by haroldsh...@gmail.com on 10 Feb 2010 at 4:01

[GoogleCodeExporter]

Didn't notice this was still open when I recently created Issue 47

Bottom line: still broken, but for a different reason.

Original comment by d...@ravenbrook.com on 8 Mar 2010 at 2:26

[GoogleCodeExporter]

Issue 47 is fixed and I'm closing this.  When it breaks again in Python 2.4 
please open a new issue.

Original comment by d...@ravenbrook.com on 16 Mar 2010 at 6:01

• Changed state: Fixed