search

uc-cdis / peregrine

GraphQL search API service
Apache License 2.0
11 stars 9 forks source link

PXD-539 ⁃ support bagit as an export format for file nodes #10

Closed philloooo closed 6 years ago

philloooo commented 6 years ago

given a graphql query, if the result nodes are files, return a bdbag with manifest for the file

trevars commented 6 years ago

Blocked by lack of information from DCP.

trevars commented 6 years ago

There are some new specifications for using BagIt.

gneglur commented 6 years ago

2: UChicago Data Explorer (windmill) → FireCloud

User logs into Windmill using Google credentials (via Fence) Who can see what data (Authz) is enforced by Windmill (via Fence) User browses TopMed data, selects a subset and clicks “Export to Firecloud (BDBag)” Windmill redirects to {firecloud}/import/bdbag?bagit_url={presigned_url} Firecloud asks user to sign in on their side and and authZ for compute resource Firecloud use the bagit url to call the bagit url and load the files into the workspace

giangbui commented 6 years ago

Spent 8 hours so far

philloooo commented 6 years ago

so we should finish it by:

  1. push the data to a s3 bucket (the key can be <userid>/<uuid><unixtime> )
  2. clean up local temp data
  3. return presigned url
giangbui commented 6 years ago

spent ~6 hours for generate presigned url