search

sibis-platform / ncanda-data-integration

This is the Data Integration, MRI, and Bioinformatics Component of the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA), funded by the NIAAA.
https://www.nitrc.org/projects/ncanda-datacore
BSD 3-Clause "New" or "Revised" License
4 stars 10 forks source link

Restricting less_content_than_max #496

Closed gcyoung1 closed 2 years ago

gcyoung1 commented 2 years ago

Do we want to exclude more forms so that only the NPs are included in the report?

gcyoung1 commented 2 years ago

From Simon - "Definitely remove anything limesurvey. Possibly restrict it to just np and biologicals"

gcyoung1 commented 2 years ago

Test run excluding limesurvey inventories:

ncanda@pipeline-back[pipeline_back_1]:/fs/ncanda-share/beta/griffin$ for file in /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/*; do echo $file; python /fs/ncanda-share/beta/griffin/ncanda-data-integration/scripts/qc/filter_inventory.py -o /fs/ncanda-share/beta/griffin/temp/uh.csv less_content_than_max -v -i $file;done
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/biological_bp.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/biological_bp.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/biological_mr.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/biological_mr.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/biological_np.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/biological_np.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/brief.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/brief.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/clinical.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/clinical.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/cnp_summary.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/cnp_summary.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/delayed_discounting_100.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/delayed_discounting_100.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/delayed_discounting_1000.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/delayed_discounting_1000.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/limesurvey_ssaga_parent.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/limesurvey_ssaga_parent.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/limesurvey_ssaga_youth.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/limesurvey_ssaga_youth.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/mr_session_report.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/mr_session_report.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/mri_report.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/mri_report.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/mri_stroop.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/mri_stroop.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_grooved_pegboard.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_grooved_pegboard.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_modified_greglygraybiel_test_of_ataxia.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_modified_greglygraybiel_test_of_ataxia.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_reyosterrieth_complex_figure.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_reyosterrieth_complex_figure.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_reyosterrieth_complex_figure_files.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_reyosterrieth_complex_figure_files.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_waisiv_coding.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_waisiv_coding.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_wrat4_word_reading_and_math_computation.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/np_wrat4_word_reading_and_math_computation.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/paced_auditory_serial_addition_test_pasat.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/paced_auditory_serial_addition_test_pasat.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/parent_report.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/parent_report.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/participant_last_use_summary.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/participant_last_use_summary.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/saliva_hormone_survey.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/saliva_hormone_survey.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/saliva_samples.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/saliva_samples.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/stroop.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/stroop.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/visit_date.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/visit_date.csv => /fs/ncanda-share/beta/griffin/temp/uh.csv
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/youth_report_1.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/youth_report_1.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/youth_report_1b.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/youth_report_1b.csv => no matches, skipping.
/fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/youth_report_2.csv
Filter less_content_than_max used on /fs/ncanda-share/log/make_all_inventories/inventory/old_inventories/6y_visit_arm_1/youth_report_2.csv => no matches, skipping.
gcyoung1 commented 2 years ago

@kipohl I think just excluding limesurvey is fine