Closed geowurster closed 9 years ago
Closes #104.
@pwoods25443 we may find that this does not completely close #104 once we actually test it, but its a good start, and worth implementing anyway.
Supports syntax like:
import gzip import gpsdio with gzip.open(path) as gz, gpsdio.open(gz, compression='GZIP', driver=<name>) as src: print(next(src))
Which we need for Luigi's file targets:
import luigi from luigi.contrib.gcs import GCSTarget import gpsdio class Convert(luigi.Task): infile = luigi.Parameter() outfile = luigi.Parameter() def input(self): return GCSTarget(self.infile) def output(self): return GCSTarget(self.outfile) def run(self): with gpsdio.open(self.input.open(), driver='NewlineJSON') as src, gpsdio.open(self.output.open('w'), 'w', driver='MsgPack', compression='GZIP') as dst: for msg in src: dst.write(msg)
Not very pretty, but it works.
Closes #104.
@pwoods25443 we may find that this does not completely close #104 once we actually test it, but its a good start, and worth implementing anyway.
Supports syntax like:
Which we need for Luigi's file targets:
Not very pretty, but it works.