New filename causes missing tables

[ypotier]

It seems the PPMI updated its file list since the scripts have been created. When running the create scripts, multiple files are not processed correctly. I have the following output showing all the files being processed but some tables are not created because of change in filename or new files.

I pasted my output of the create script below.

List of processed files

Processing Prodromal_Diagnostic_Questionnaire.csv Processing FBB_Analysis_Data.csv Processing University_of_Pennsylvania_Smell_ID_Test.csv Processing FOUND_RFQ_Calcium_Channel_Blockers.csv Processing Lexical_Fluency.csv Processing Wearable_and_Sensor_Eligibility.csv Processing Features_of_REM_Behavior_Disorder.csv Processing Epworth_Sleepiness_Scale.csv Processing Letter_-_Number_Sequencing__PD_.csv Processing Florbetaben_Eligibility.csv Processing AV-133_Image_Metadata.csv Processing Diagnostic_Features.csv Processing Clinical_Diagnosis_and_Management.csv Processing FOUND_RFQ_Height___Weight.csv Processing Clinical_Labs.csv Processing Skin_Biopsy.csv Processing Page_Descriptions.csv Processing Determination_of_Falls.csv Processing General_Medical_History.csv Processing Contact_Information_Brain_Bank.csv Processing Upper_Extremity_Function-_Short_Form.csv Processing MDS_UPDRS_Part_I__Patient_Questionnaire.csv Processing Use_of_PD_Medication.csv Processing Participation_in_Other_Clinical_Trials.csv Processing AV-133_Imaging.csv Processing TAP-PD_OPDM_Use_Questionnaire.csv Processing Conclusion_of_Study_Participation.csv Processing ST_CATALOG.csv Processing Skin_Biopsy_Eligibility.csv Processing RBD_PSG_Eligibility.csv Processing DaTscan_Imaging.csv Processing SPECT_Scan_Information_Source_Document.csv Processing Olfactory_UPSIT.csv Processing General_Physical_Exam.csv Processing FOUND_RFQ_Head_Injury.csv Processing DATScan_Analysis.csv Processing Concomitant_Medications.csv Processing Inclusion_Exclusion.csv Processing Current_Biospecimen_Analysis_Results.csv Processing AV-133_Subject_Eligibility.csv Processing Genetic_Testing_Results.csv Processing General_Neurological_Exam.csv Processing Primary_Diagnosis.csv Processing Whole_Blood_Sample_Collection.csv Processing REM_Sleep_Disorder_Questionnaire.csv Processing Patient_Status.csv Processing Cognitive_Assessments.csv Processing FOUND_RFQ_Pesticides_at_Work.csv Processing FOUND_RFQ_Caffeine.csv Processing TAP-PD_Conclusion.csv Processing Socio-Economics.csv Processing FOUND_RFQ_Occupation.csv Processing FOUND_RFQ_Female_Reproductive_Health.csv Processing MDS_UPDRS_Part_I.csv Processing Current_Medical_Conditions_Log.csv Processing PASE_-_Household_Activity.csv Processing iPSC_Blood_Sample.csv Processing FOUND_RFQ_Smoking_History.csv Processing Gait_Data___Arm_swing.csv Processing Adverse_Event_Log.csv Processing Pilot_Biospecimen_Analysis_Results.csv Processing MDS-UPDRS_Tagging_Status.csv Processing Family_History__PD_.csv Processing Initiation_of_PD_Medication-_incidents.csv Processing Code_List.csv Processing Modified_Boston_Naming_Test.csv Processing MDS_UPDRS_Part_IV.csv Processing Signature_Form.csv Processing Laboratory_Procedures.csv Processing Communication-Short_Form.csv Processing SCOPA-AUT.csv Processing Cognitive_Categorization.csv Processing Screening___Demographics.csv Processing Laboratory_Procedures_with_Elapsed_Times.csv Processing Modified_Schwab_+_England_ADL.csv Processing FOUND_RFQ_Dictionary.csv Processing Hopkins_Verbal_Learning_Test.csv Processing FOUND_Enrollment_Status.csv Processing AV-133_SBR_Results.csv Processing Lumbar_Puncture_Sample_Collection.csv Processing Montreal_Cognitive_Assessment__MoCA_.csv Processing IUSM_BIOSPECIMEN_CELL_CATALOG.csv Processing 151_Project_Extra___Buffer_and_Calibrator_Data.csv Processing Lower_Extremity_Function__Mobility_.csv Processing Derived_Variable_Definitions_and_Score_Calculations.csv Processing Semantic_Fluency.csv Processing MDS_UPDRS_Part_III.csv Processing Deprecated_Biospecimen_Analysis_Results.csv Processing Geriatric_Depression_Scale__Short_.csv Processing Trail_Making_A_and_B.csv Processing Blood_Chemistry___Hematology.csv Processing Device_Placement_Log.csv Processing Wearable_and_Sensor_Participation_Status.csv Processing TAP-PD_OPDM_Assessment.csv Processing Symbol_Digit_Modalities.csv Processing State-Trait_Anxiety_Inventory.csv Processing FOUND_RFQ_Alcohol.csv Processing TAP-PD_Kinetics_Device_Testing.csv Processing DNA_Sample_Collection.csv Processing Center-Subject_List.csv Processing Data_Dictionary.csv Processing Contact_Information_FOUND.csv Processing FOUND_Self-Reported_Dx.csv Processing Pregnancy_Form.csv Processing Benton_Judgment_of_Line_Orientation.csv Processing Research_Advance_Directive.csv Processing FOUND_RFQ_Pesticides_Non-Work.csv Processing Magnetic_Resonance_Imaging.csv Processing FOUND_RFQ_Toxicant_History.csv Processing PASE_-_Leisure_Time_Activity.csv Processing Cognition_Function-Short_Form.csv Processing FOUND_RFQ_Anti-Inflammatory_Meds.csv Processing MDS_UPDRS_Part_II__Patient_Questionnaire.csv Processing TAP-PD_Subject_Eligibility.csv Processing Surgery_for_Parkinson_Disease.csv Processing Randomization_table.csv Processing FOUND_RFQ_Residential_History.csv Processing Neurological_Exam_-_Cranial_Nerves.csv Processing DTI_Regions_of_Interest.csv Processing Genetic_and_Registry_Diagnostic_Questionnaire.csv Processing PD_Features.csv Processing FOUND_RFQ_Physical_Activity.csv Processing Vital_Signs.csv Processing DaTSCAN_SPECT_Visual_Interpretation_Assessment.csv Processing IUSM_CATALOG.csv Processing FBB_Metadata.csv Processing MRI_Imaging_Data_Transfer_Information_Source_Document.csv Processing QUIP_Current_Short.csv

	basename	pat_id	pag_cnt	table_name
0	Prodromal_Diagnostic_Questionnaire.csv	True	2	None
3	FOUND_RFQ_Calcium_Channel_Blockers.csv	False	0	None
13	FOUND_RFQ_Height___Weight.csv	False	0	None
22	Use_of_PD_Medication.csv	True	2	None
34	FOUND_RFQ_Head_Injury.csv	False	0	None
38	Current_Biospecimen_Analysis_Results.csv	True	0	None
42	Primary_Diagnosis.csv	True	3	None
46	Cognitive_Assessments.csv	True	2	None
47	FOUND_RFQ_Pesticides_at_Work.csv	False	0	None
48	FOUND_RFQ_Caffeine.csv	False	0	None
51	FOUND_RFQ_Occupation.csv	False	0	None
52	FOUND_RFQ_Female_Reproductive_Health.csv	False	0	None
57	FOUND_RFQ_Smoking_History.csv	False	0	None
58	Gait_Data___Arm_swing.csv	True	0	None
60	Pilot_Biospecimen_Analysis_Results.csv	True	0	None
75	FOUND_RFQ_Dictionary.csv	False	0	None
77	FOUND_Enrollment_Status.csv	True	0	None
82	151_Project_Extra___Buffer_and_Calibrator_Data...	False	0	None
84	Derived_Variable_Definitions_and_Score_Calcula...	False	0	None
85	Semantic_Fluency.csv	True	2	None
86	MDS_UPDRS_Part_III.csv	True	4	None
87	Deprecated_Biospecimen_Analysis_Results.csv	True	0	None
96	FOUND_RFQ_Alcohol.csv	False	0	None
102	FOUND_Self-Reported_Dx.csv	True	0	None
106	FOUND_RFQ_Pesticides_Non-Work.csv	False	0	None
108	FOUND_RFQ_Toxicant_History.csv	False	0	None
111	FOUND_RFQ_Anti-Inflammatory_Meds.csv	False	0	None
116	FOUND_RFQ_Residential_History.csv	False	0	None
121	FOUND_RFQ_Physical_Activity.csv	False	0	None
122	Vital_Signs.csv	True	2	None

[sameerd]

Thanks for reporting. I updated the code to fix a few of these collisions.