planetarypy / planetaryimage

Python PDS and Isis Cube file parser.
BSD 3-Clause "New" or "Revised" License
39 stars 20 forks source link

Added a support for gzipped fie in planetaryimage. #43

Closed bvnayak closed 8 years ago

bvnayak commented 8 years ago

refs #42

wtolson commented 8 years ago

It might be nice to implement an interface for handling compression similar to pandas.read_csv(). It takes the following compression argument:

compression: decompress 'gzip' and 'bz2' formats on the fly. Set to 'infer' (the default) to guess a format based on the file extension.

This will allow additional compression formats like bz2 from the stdlib as well as allowing the user to override and disable the compression.

godber commented 8 years ago

@wtolson your suggestion of catching an IOError was a good idea but an error was not being raised, fromfile() was simply returning a null array. So we opted for the explicit route.

Now we just have to figure out how to deal with the python 2.6 and 2.7 issues.

wtolson commented 8 years ago

Interesting, what behaviour are you seeing? On python2.7/numpy1.9.2 I'm seeing this:

In [1]: import gzip

In [2]: import numpy

In [3]: with gzip.open('test.txt.gz', 'rb') as fp:
    data = numpy.fromfile(fp, int, 10)
   ...:     
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-3-74cf4401a888> in <module>()
      1 with gzip.open('test.txt.gz', 'rb') as fp:
----> 2     data = numpy.fromfile(fp, int, 10)
      3 

IOError: first argument must be an open file

In [4]: print numpy.version.full_version
1.9.2
godber commented 8 years ago

I think Bhavin is workin in python 3 by default. He's in class at the moment.

bvnayak commented 8 years ago

GZIP: Changed in version 2.7: Support for the with statement was added.

BZ2.open is new in Version 3.3