ACED-IDP / aced-idp.github.io

Documentation Site for the ACED Data Commons
https://aced-idp.github.io/
Apache License 2.0
1 stars 0 forks source link

Update documentation for new command line structure #7

Closed bwalsh closed 6 months ago

bwalsh commented 6 months ago

This issue documents command line changes: https://github.com/ACED-IDP/gen3_util/issues/34

The changes are implemented in this branch: https://github.com/ACED-IDP/gen3_util/pull/39

This site should be updated to reflect those changes.

bwalsh commented 6 months ago

For example: gen3_util files manifest put is now gen3_util files push. gen3_util meta export is now gen3_util meta pull

A full list of the changes are on the tests/integration/test_meta.py here:

https://github.com/ACED-IDP/gen3_util/compare/development...feature/refactor-cli#diff-fcba4eb98b2de807df96f9cd2f09f26f9aa7c9acfed0feee3bc59800844a71dd

bwalsh commented 6 months ago

@jordan2lee @kellrott @teslajoy

bwalsh commented 6 months ago

Example workflow:

# create this project
export PROJECT_ID=test-test2
# on this cluster
export GEN3_UTIL_PROFILE=local

# create the project creation requests
gen3_util projects new
# and sign them
gen3_util access sign

# add files to the project, associated with the given patient and specimen ids
gen3_util files add tests/fixtures/add_files_to_study/sub-dir/file-4.tsv --patient_id P1 --specimen_id S1
gen3_util files add tests/fixtures/add_files_to_study/sub-dir/file-3.pdf --patient_id P1 --specimen_id S2
gen3_util files add tests/fixtures/add_files_to_study/sub-dir/file-5 --patient_id P1 --specimen_id S3
gen3_util files add tests/fixtures/add_files_to_study/file-1.txt --patient_id P2 --specimen_id S4
gen3_util files add tests/fixtures/add_files_to_study/README.md
gen3_util files add tests/fixtures/add_files_to_study/file-2.csv --patient_id P3 --specimen_id S5

# upload the files, and by default create metadata for them
gen3_util files push

# pull down the metadata just created and check it
gen3_util meta pull /tmp/$PROJECT_ID --force

ls -1 /tmp/$PROJECT_ID
DocumentReference.ndjson
Patient.ndjson
ResearchStudy.ndjson
ResearchSubject.ndjson
Specimen.ndjson

jq -rc '[.id, .identifier[0].value ]'  /tmp/$PROJECT_ID/Patient.ndjson
["585c97dc-02f3-5089-9885-9a893fd21fc5","P1"]
["4661d3b8-99bd-54bf-8f65-5c46472a8909","P2"]
["947a037b-a702-5510-b989-fd2af4f7cfdb","P3"]

jq -rc '[.id, .identifier[0].value, .subject.reference]'  /tmp/$PROJECT_ID/Specimen.ndjson
["0a8e392c-9cef-5fbf-82cd-a95f9ed4d4f9","S1","Patient/585c97dc-02f3-5089-9885-9a893fd21fc5"]
["32d36e16-5ae8-5932-94ba-ed4e83e0774f","S2","Patient/585c97dc-02f3-5089-9885-9a893fd21fc5"]
["fe317dea-51f0-59ce-ba44-d8824ac36f58","S3","Patient/585c97dc-02f3-5089-9885-9a893fd21fc5"]
["8f5721ba-a8bb-5637-ac93-f1180ea6099b","S4","Patient/4661d3b8-99bd-54bf-8f65-5c46472a8909"]
["4030953a-e519-5141-9d92-6cadd0e2c635","S5","Patient/947a037b-a702-5510-b989-fd2af4f7cfdb"]

jq -rc  '. | [.subject.reference, .content[0].attachment.url] '  /tmp/test-test2/DocumentReference.ndjson 
["Specimen/0a8e392c-9cef-5fbf-82cd-a95f9ed4d4f9","file:///tests/fixtures/add_files_to_study/sub-dir/file-4.tsv"]
["Specimen/32d36e16-5ae8-5932-94ba-ed4e83e0774f","file:///tests/fixtures/add_files_to_study/sub-dir/file-3.pdf"]
["Specimen/fe317dea-51f0-59ce-ba44-d8824ac36f58","file:///tests/fixtures/add_files_to_study/sub-dir/file-5"]
["Specimen/8f5721ba-a8bb-5637-ac93-f1180ea6099b","file:///tests/fixtures/add_files_to_study/file-1.txt"]
["ResearchStudy/bec49ccc-d528-5c01-b545-d3b7e09345df","file:///tests/fixtures/add_files_to_study/README.md"]
["Specimen/4030953a-e519-5141-9d92-6cadd0e2c635","file:///tests/fixtures/add_files_to_study/file-2.csv"]
bwalsh commented 6 months ago

Merged