AllenCell / cvapipe

Cell Variance Analysis Pipeline
Other
2 stars 0 forks source link

admin/update-datastep-version #22

Closed evamaxfield closed 4 years ago

evamaxfield commented 4 years ago

This updates the version pin of datastep to 0.0.1. Copying the comment from the code here:

    # SPECIAL RELEASE OF DATASTEP TO RESOLVE MALFORMED MANIFEST
    # https://github.com/AllenCellModeling/datastep/tree/special-release
    #
    # Fortunately we left the 0.0.* series of releases empty
    # New datastep projects will by default use the 0.1.* series of releases

Files pushed to quilt on this version of datastep will be prefixed with a hash of their absolute path. See quilt3distribute for examples of this behavior.

This occurred because @jxchen01 had paths following the structure: unique_part/general_part.tiff, i.e. cell_id/seg.tiff instead of the other way around: general_part/unique_part.tiff, i.e. segs/cell_id.tiff.

(If the column name for this file in the manifest was "cell_segs", this would result in: cell_segs/seg.tiff for all files... All files in the manifest are reduced to a single reference which means nothing but a single file is uploaded.)

In the future, can all devs on this project please push a sample of their data to quilt prior to running the entire dataset AND it is preferred to have the unique part of a file in the file name.

This was a design decision made a long time ago to allow the removal of the prefix hash.