clowder-framework / clowder

A data management system that allows users to share, annotate, organize and analyze large collections of datasets. It provides support for extensible metadata annotation using JSON-LD and a distribute analytics event bus for automatic curation of uploaded data.
https://clowderframework.org/
University of Illinois/NCSA Open Source License
35 stars 17 forks source link

download dataset/collection as bagit #319

Closed robkooper closed 2 years ago

robkooper commented 2 years ago

there is now an optional parameter to download dataset/collection as a bagit zip file.

Description

Review Time Estimate

Types of changes

Checklist:

robkooper commented 2 years ago

To test this:

  1. create a dataset
  2. add one or more files/folders to the dataset
  3. create collection
  4. add one ore more datasets + collections to collection

download using curl:

curl -o bagit.zip http://localhost:9000/api/collections/YOURID/download\?bagit\=false\&compression\=1\&key\=r1ek3rs

output looks approximately like:

Archive:  bagit.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
  3729064  01-30-2022 22:11   test_col/test_ds/folder1//DJI_0034.JPG
  3977058  01-30-2022 22:11   test_col/test_ds/folder1//DJI_0035.JPG
---------                     -------
  7706122                     2 files

change the bagit=true to download as bagit file and output looks like:


Archive:  bagit.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
      334  01-30-2022 22:22   test_col/test_col_info.json
       70  01-30-2022 22:22   test_col/test_col_metadata.json
      285  01-30-2022 22:22   test_col/test_ds/test_ds_info.json
        3  01-30-2022 22:22   test_col/test_ds/test_ds_metadata.json
      307  01-30-2022 22:22   test_col/test_ds/folder1//DJI_0034.JPG_info.json
        3  01-30-2022 22:22   test_col/test_ds/folder1//DJI_0034.JPG_metadata.json
  3729064  01-30-2022 22:22   test_col/test_ds/folder1//DJI_0034.JPG
      307  01-30-2022 22:22   test_col/test_ds/folder1//DJI_0035.JPG_info.json
        3  01-30-2022 22:22   test_col/test_ds/folder1//DJI_0035.JPG_metadata.json
  3977058  01-30-2022 22:22   test_col/test_ds/folder1//DJI_0035.JPG
       55  01-30-2022 22:22   test_col/bag-info.txt
      250  01-30-2022 22:22   test_col/bagit.txt
      524  01-30-2022 22:22   test_col/manifest-md5.txt
      139  01-30-2022 22:22   test_col/tagmanifest-md5.txt
---------                     -------
  7708402                     14 files
tcnichol commented 2 years ago

I have tested this and it works as described. Marking this approved.