valohai / valohai-utils

Python helper library for Valohai
MIT License
2 stars 2 forks source link

.compess() should be able to compress subdirectories also #58

Closed drazendee closed 3 years ago

drazendee commented 3 years ago

Today compress() allows users to compress a set of files to an individual file.

valohai.outputs('resized').compress("*.png", "images.zip", remove_originals=True)

However, it doesn't allow you to compress all subfolders and their files from the output. For example:

- myoutput
  - train
    - images
    - labels
  - eval
    - images
    - labels

There are several cases where you'd output files into a specific folder structure, and then compress that and upload to your cloud storage (and maintain the folder structure for when you pull it down for the next execution)