stain
commented
5 years ago Thank you for submitting to RO2019's open peer review process. We will shortly be assigning members from the Programme Committee to review.
Feel free to respond to reviewers comments and to update the submission if needed.
Tip: Anyone is welcome to add an informal review below using GitHub comments; as an author perhaps you would volunteer to review one of the other open submissions?
Reviewers, please copy this review form and add as a comment. You don't need to use this form if you are not assigned from the PC.
## Quality of Writing
_Is the text easy to follow? Are core concepts defined or referenced?
Is it clear what is the author's contribution?_
(delete as appropriate)
* excellent / good / fair / poor
## Research Object / Zenodo
_URL for a Research Object or Zenodo record provided?
Guidelines <http://researchobject.org/ro2019/submitting> followed?
Open format (e.g. HTML)?
Sufficient metadata, e.g. links to software?
Some form of Data Package provided?
Add text below if you need to clarify your score._
(delete as appropriate)
* none (e.g. only abstract in easychair/github)
* basic (e.g. Zenodo with PDF and minimal metadata)
* sufficient (e.g. HTML, detailed Zenodo metadata)
* good (followed guidelines, demonstrating own format, related resources included, but some details missing)
* excellent (e.g. followed all guidelines, complete metadata or RO-like research data package, linked data, provenance)
## Overall evaluation
_Please provide a brief review, including a justification for your scores.
Both score and review text are required._
(delete as appropriate)
* strong reject
* reject
* weak reject
* borderline
* weak accept
* accept
* strong acceptFor confidential remarks or questions about the peer-review process, contact ro2019@easychair.org
dakoop
Authors
Keywords
Homepage
https://github.com/AllenCellModeling/quilt3distribute
Abstract
A core principal of research is the affordability and ease of reproducing the results found by an experiment and to minimize the challenge of experimental reproducibility, it is common for researchers to share the dataset used to produce the results of an experiment. Methods for managing and distributing these datasets however, are ill-suited for imaging datasets, or more generally: large object datasets, because they commonly resemble a manifest and require additional packaging and organization than their feature set counterparts.
Quilt3Distribute (Q3D) is a software application that enables the distribution of manifest style datasets which can be made of up thousands of individual files.
Full Abstract Available at:
(shown below)
Title: Managing Manifests and Distributing Datasets Date: 09.01.2019 Author: Jackson Brown, Allen Institute for Cell Science Corresponding Author Email: jacksonb@alleninstitute.org
Managing Manifests and Distributing Datasets
Abstract
A core principal of research is the affordability and ease of reproducing the results found by an experiment and to minimize the challenge of experimental reproducibility, it is common for researchers to share the dataset used to produce the results of an experiment. Methods for managing and distributing these datasets however, are ill-suited for imaging datasets, or more generally: large object datasets, because they commonly resemble a manifest and require additional packaging and organization than their feature set counterparts.
Quilt3Distribute (Q3D) is a software application that enables the distribution of manifest style datasets which can be made of up thousands of individual files.
Manifests
There are many ways to store a dataset made up of thousands of files, but often these options have significant associated costs. One such method of packaging and distributing these datasets made popular in recent years, is the containerization of data [2, 4]. Due to the nature of geospatial data, scientists and engineers have been creating file formats, data packaging tools, and distribution systems for many years to solve these exact problems which have ultimately established standards for the packaging and distribution of their fields datasets [4]. Following suit, our proposal for storage and distribution of manifest style datasets uses a software package that falls under the containerization umbrella. "Quilt" is a software package that adds a number of features to current state of the art storage systems (e.g. Amazon's S3) [3]. Of note: Quilt performs versioning of dataset containers (what Quilt refers to as data packages), which is a desirable feature for the scientific resource stack as versioning is critical for reproducibility.
Implementation
This project expands on state of the art methods by specifically targeting the distribution of manifest style datasets. For most scientists at the Allen Institute for Cell Science, datasets are commonly tabular, with a column, or multiple columns, filled with file paths to larger resources (images). The rest of the tabular dataset is largely metadata about the files referenced in each row. To expand on the state of the art, our Python package attaches the metadata from columns found in a tabular manifest style dataset to each file found in the manifest before distributing them with Quilt.
To help the end user of the dataset, our package additionally cleans and standardizes the metadata it finds. A few examples of these cleaning and validation behaviors are:
1) Asserting or casting all values in a metadata column are that of the most common data type for that metadata column. (Ex: if the values "4" (integer four), "1" (integer one), and "'4'" (string four) exist in the same column, the application attempts to cast the instance of the string four to the columns most common data type: an integer)
2) Enforcing that every file path found exists prior to attempting distribution. (Commonly, when collaboratively working on shared datasets, files will move to a new location during the process of development.)
Learning from the extensive work on linked data, retention of file associations is an incredibly valuable tool for not only navigation between objects but also the relationships between those objects [1]. To apply these linked data concepts in our own application, if a manifest has multiple columns that contain file paths to resources, our application additionally constructs a lookup table for those shared file associations and places the relevant portion of the lookup table in each file's metadata. An illustrative example of this behavior is when an imaging manifest has a column for full size images and a column for thumbnail images. For each row of the manifest, the application makes an entry in the Quilt package that pairs the full size image to the smaller thumbnail image by storing the read path to the other in the metadata for each member of the pair.
Quilt3Distribute (Q3D) is an open-source Python application and additional details and documentation can be found at: https://github.com/AllenCellModeling/quilt3distribute. Q3D is currently used as a primary production system to distribute terabytes of imaging datasets from the Allen Institute for Cell Science.
Acknowledgements
We wish to thank the Allen Institute for Cell Science founder, Paul G. Allen, for his vision, encouragement, and support. This work could not have been completed without the additional support and input from all members of the AllenInstitute for Cell Science modeling team.
References