As a release manager , I want a test script to ensure comprehensive and repeatable testing of the new feature(s).
Use case
As a testing engineer, I want a data submission, validation, and upload script to ensure accurate and secure processing of user-submitted data.
Definition of Done:
[x] Test script is created and documented.
[ ] The script is reviewed and approved by the testing team.
[ ] optional The script is integrated into the testing process and automated frameworks.
[ ] script is executed and all acceptance criteria are met.
Considerations
[ ] The system should generate clear error messages for users in case of invalid data submissions, guiding them on how to correct the issues.
[ ] It should perform thorough validation of data integrity to prevent corruption or loss during the upload process.
[ ] Security measures should be implemented to protect against potential data breaches or unauthorized access during the submission and upload process.
[ ] It must log relevant information, including successful uploads and any errors encountered, for auditing and debugging purposes.
[ ] The system should be version-controlled to track changes and updates over time.
[ ] The validation and upload process should be easily integrable into automated testing frameworks for continuous integration.
Script
submitter test script
# Use case: As a data submitter, I will need to create a project.
## test should work with or without environment variables
#export G3T_PROFILE=local
#export G3T_PROJECT_ID=ohsu-test002b
#g3t init
unset G3T_PROJECT_ID
unset G3T_PROFILE
g3t --profile local init ohsu-test001b
# Use case: As a institution data steward, I need to approve the project before it can be shared.
g3t utilities access sign
# Use case: As a ACED administrator, I need to create projects in sheepdog so that submissions can take place.
g3t utilities projects ls
## test: the project should be listed as incomplete
g3t utilities projects create
## test: the project should be listed as complete
# Use case: As a data submitter, I will need to add files to the project and associate them with a subject(patient).
g3t add tests/fixtures/dir_to_study/file-1.txt --patient P1
g3t utilities meta create
## test meta generation: META should have 4 files
g3t commit -m "commit-1"
## test that the commit: g3t status should return commit info - was message added?
# resource_counts:
# DocumentReference: 1
# Patient: 1
# ResearchStudy: 1
# ResearchSubject: 1
# Use case: when subjects are added to study I need to add them to the project.
g3t add tests/fixtures/dir_to_study/file-2.csv --patient P2
g3t status
## test add: should return one entry in "uncommitted_manifest:"
g3t utilities meta create
## test meta generation: META should have 4 files Patient ResearchSubject DocumentReference should have 1 new record each
g3t commit -m "commit-2"
## test the commit: g3t status should return commit info - was message added? there should only be the three new records
# resource_counts:
# DocumentReference: 1
# Patient: 1
# ResearchSubject: 1
# manifest_files:
# - tests/fixtures/dir_to_study/file-2.csv
# Use case: some subjects have specimens, I need to add them to the project.
g3t add tests/fixtures/dir_to_study/sub-dir/file-3.pdf --patient P3 --specimen S3
g3t utilities meta create
## test should create a Specimen.ndjson file in META
# Created 4 new records.
wc -l META/Specimen.ndjson
# 1 META/Specimen.ndjson
g3t diff
## test diff: should show new records
g3t commit -m "commit-3"
## test the commit: g3t status should return commit info - was message added? 4 new records
# message: commit-3
# resource_counts:
# DocumentReference: 1
# Patient: 1
# ResearchSubject: 1
# Specimen: 1
# manifest_files:
# - tests/fixtures/dir_to_study/sub-dir/file-3.pdf
# Use case: I'm ready to share my data
## push to remote
g3t push
## test: the system should respond with reasonable, informative messages without too much verbosity
## I need to know the status of my project. During job execution, I should be able to query the status.
g3t status
## test: After job execution, I should have detailed information about the results.
# pushed_commits:
# - published_timestamp: 2024-01-19T09:45:47.018426
# published_job:
# output:
# uid: 82322961-8d2a-47e4-8833-af0e299aa393
# name: fhir-import-export-ohiwi
# status: Completed
# commits:
# - d050c8f931bab152279ff18e0a21434f commit-1
# - 2f77cf6017ec3b0485b7493ebe459f53 commit-2
# - a550281b43713937ce684e3cab13639f commit-3
## test: Once complete, the remote counts should reconcile with my activity
#remote:
# resource_counts:
# DocumentReference: 3
# Patient: 3
# ResearchStudy: 1
# ResearchSubject: 3
# Specimen: 1
wc -l META/*.ndjson
# 3 META/DocumentReference.ndjson
# 3 META/Patient.ndjson
# 1 META/ResearchStudy.ndjson
# 3 META/ResearchSubject.ndjson
# 1 META/Specimen.ndjson
## If I want more detailed information, I should be able to query it
## get UID from status -> local.pushed_commits.published_job.output.uid
g3t utilities jobs get UID
# ....
# Use case: As a data submitter, when I know more about meta, I should be able to add it.
# e.g. alter a patient record
sed -i.bak 's/"P1"}]}/"P1"}], "gender": "male"}/' META/Patient.ndjson
# see https://stackoverflow.com/a/22084103
rm META/Patient.ndjson.bak
g3t diff
## test diff: should show changed records
g3t commit -m "commit-4"
## test: the commit should process only one patient record
#resource_counts:
# Patient: 1
## Use case: I should be able to publish a 'meta only' change
g3t push
## Use case: As a human being, I make mistakes, the system should prevent me from committing `no changes`
g3t commit -m "commit-5 has no changes"
## test: the system should reject the commit
# msg: No resources changed in META
## Use case: As a human being, I make mistakes, the system should prevent me from committing `invalid fhir`
sed -i.bak 's/"gender"/"foobar"/' META/Patient.ndjson
# see https://stackoverflow.com/a/22084103
rm META/Patient.ndjson.bak
g3t commit -m "commit-6 has invalid fhir"
## test: should fail validation, the response should be informative and give me enough information to fix the problem
Epic
As a release manager , I want a test script to ensure comprehensive and repeatable testing of the new feature(s).
Use case
As a testing engineer, I want a data submission, validation, and upload script to ensure accurate and secure processing of user-submitted data.
Definition of Done:
Considerations
Script
submitter test script