seung-lab / cloud-files

Threaded Python and CLI client library for AWS S3, Google Cloud Storage (GCS), in-memory, and the local filesystem.
BSD 3-Clause "New" or "Revised" License
38 stars 8 forks source link

feat: add transcode to compression #15

Closed william-silversmith closed 4 years ago

william-silversmith commented 4 years ago
cloudfiles/compression.py

def transcode(
  files, encoding, level=None, 
  progress=False, in_place=False
):
  """
  Given the output of get, which may include raw files, 
  transcode the compresson scheme into the one specified by 
  encoding. If the files are raw and the schemes match, 
  the decompression-compression cycle will be skipped.

  level: input of compression level e.g. 6 for gzip
  progress: show a progress bar
  in_place: modify the files in-place to save memory

  Yields: {
    'path': ,
    'content': ,
    'compress': ,
    'raw': ,
  }