search

Clarivate-LSPS / tMDataLoader

new Groovy-based tranSMART ETL
8 stars 19 forks source link

uploading 2 tables at once discard one #63

Closed kforner closed 7 years ago

kforner commented 7 years ago

Hello,

I encountered a behaviour that I do not understand. I have 2 data files, if I upload them one by one (with merge_mode APPEND) all data is properly uploaded, but if try to upload them in one tMDataLoader run, my second table seems to discarded. What am I missing ? Thanks.

two at once:

$output
 [1] ">>> USING CONFIG: Config.groovy"                                                                                                
 [2] ">>> USING INTERACTIVE MODE"                                                                                                     
 [3] ">>> WILL STOP ON FAIL"                                                                                                          
 [4] ">>> OVERRIDING CONFIG: VISIT_NAME first"                                                                                        
 [5] ">>> FYI: using VISIT_NAME before DATA_VALUE as default behavior (per config or command line)"                                   
 [6] "06/12 15:55:38 MSG !!! TM_ETL VERSION 1.2.4-SNAPSHOT"                                                                           
 [7] "06/12 15:55:38 MSG ==== STARTED ===="                                                                                           
 [8] "06/12 15:55:38 MSG Using directory: ETL"                                                                                        
 [9] "06/12 15:55:38 MSG Processing directory: ETL"                                                                                   
[10] "06/12 15:55:38 MSG Processing directory: ETL/Inception"                                                                         
[11] "06/12 15:55:38 MSG === PROCESSING STUDIES IN '\\Inception' ==="                                                                 
[12] "06/12 15:55:38 MSG == Found study: Low Dimensional Data =="                                                                     
[13] "06/12 15:55:38 MSG Processing Clinical data"                                                                                    
[14] "06/12 15:55:38 MSG Connecting to database server"                                                                               
[15] "06/12 15:55:38 MSG Mapping file: AD2_Mapping_File.txt"                                                                          
[16] "06/12 15:55:38 MSG Processing Comorbidity.txt"                                                                                  
[17] "06/12 15:55:39 MSG Processed 376 rows"                                                                                          
[18] "06/12 15:55:39 MSG Processing Medication.txt"                                                                                   
[19] "06/12 15:55:39 MSG Processed 373 rows"                                                                                          
[20] "06/12 15:55:39 MSG Job ID: 5083"                                                                                                
[21] "06/12 15:55:39 MSG Run procedures: I2B2_LOAD_CLINICAL_DATA"                                                                     
[22] "06/12 15:55:39 MSG Study ID=AD2; Node=\\Inception\\Low Dimensional Data\\"                                                      
[23] "06/12 15:55:40 DBG -- Start i2b2_load_clinical_data for AD2 [Done / 0 recs / 0.000s]"                                           
[24] "06/12 15:55:40 DBG -- Load lt_src_clinical_data to work table [Done / 7520 recs / 0.085s]"                                      
[25] "06/12 15:55:40 DBG -- Set columns in wrk_clinical_data [Done / 7520 recs / 0.172s]"                                             
[26] "06/12 15:55:40 DBG -- Delete null data_values in wrk_clinical_data [Done / 0 recs / 0.007s]"                                    
[27] "06/12 15:55:40 DBG -- Remove pipes in data_value [Done / 0 recs / 0.007s]"                                                      
[28] "06/12 15:55:40 DBG -- Remove empty parentheses 1 [Done / 0 recs / 0.008s]"                                                      
[29] "06/12 15:55:40 DBG -- Remove empty parentheses 2 [Done / 0 recs / 0.007s]"                                                      
[30] "06/12 15:55:40 DBG -- Replace pipes with comma in data_label [Done / 0 recs / 0.007s]"                                          
[31] "06/12 15:55:40 DBG -- Set single visit_name to null [Done / 0 recs / 0.274s]"                                                   
[32] "06/12 15:55:40 DBG -- Remove tag markers [Done / 0 recs / 0.007s]"                                                              
[33] "06/12 15:55:40 DBG -- Set data_label to null when found in category_path [Done / 0 recs / 0.503s]"                              
[34] "06/12 15:55:41 DBG -- Remove leading, trailing, double spaces [Done / 7520 recs / 0.371s]"                                      
[35] "06/12 15:55:41 DBG -- Set visit_name to null if VISITNAME not in category_path [Done / 0 recs / 0.008s]"                        
[36] "06/12 15:55:41 DBG -- Set data_label to null if DATALABEL not in category_path [Done / 0 recs / 0.008s]"                        
[37] "06/12 15:55:41 DBG -- Set visit_name to null when found in data_label [Done / 0 recs / 0.009s]"                                 
[38] "06/12 15:55:41 DBG -- Set visit_name to null when found in data_value [Done / 0 recs / 0.009s]"                                 
[39] "06/12 15:55:41 DBG -- Insert numeric data into WZ wt_num_data_types [Done / 0 recs / 0.188s]"                                   
[40] "06/12 15:55:41 DBG -- Updated data_type flag for numeric data_types [Done / 0 recs / 0.007s]"                                   
[41] "06/12 15:55:41 DBG -- Add if missing DATALABEL, VISITNAME and DATAVALUE to category_path [Done / 7520 recs / 0.162s]"           
[42] "06/12 15:55:41 DBG -- Remove duplicates from wrk_clinical_data [Done / 0 recs / 0.015s]"                                        
[43] "06/12 15:55:41 DBG -- Check for duplicate key columns [Done / 0 recs / 0.011s]"                                                 
[44] "06/12 15:55:41 DBG -- Check for missing visit_names for subject_id/category/label/value  [Done / 1 recs / 0.014s]"              
[45] "06/12 15:55:42 DBG -- Create leaf nodes for trial [Done / 126 recs / 0.639s]"                                                   
[46] "06/12 15:55:42 DBG -- Updated last path component for timestamps [Done / 0 recs / 0.006s]"                                      
[47] "06/12 15:55:42 DBG -- Updated node name for leaf nodes [Done / 126 recs / 0.024s]"                                              
[48] "06/12 15:55:42 DBG -- Insert subject information into temp table [Done / 178 recs / 0.032s]"                                    
[49] "06/12 15:55:42 DBG -- Delete dropped subjects from patient_dimension [Done / 0 recs / 0.011s]"                                  
[50] "06/12 15:55:42 DBG -- Update subjects with changed demographics in patient_dimension [Done / 178 recs / 0.007s]"                
[51] "06/12 15:55:42 DBG -- Insert new subjects into patient_dimension [Done / 0 recs / 0.006s]"                                      
[52] "06/12 15:55:42 DBG -- Inserted new leaf nodes into I2B2DEMODATA concept_dimension [Done / 0 recs / 0.007s]"                     
[53] "06/12 15:55:42 DBG -- Updated name and data type in i2b2 if changed [Done / 126 recs / 0.021s]"                                 
[54] "06/12 15:55:42 DBG -- Inserted leaf nodes into I2B2METADATA i2b2 [Done / 0 recs / 0.007s]"                                      
[55] "06/12 15:55:42 DBG -- Delete clinical data for study from observation_fact [Done / 7698 recs / 0.031s]"                         
[56] "06/12 15:55:50 DBG -- Collect observation facts [Done / 7520 recs / 8.249s]"                                                    
[57] "06/12 15:55:50 DBG -- Drop observation facts indexes [Done / 0 recs / 0.010s]"                                                  
[58] "06/12 15:55:50 DBG -- Insert trial into I2B2DEMODATA observation_fact [Done / 7520 recs / 0.145s]"                              
[59] "06/12 15:55:50 DBG -- Inserted into I2B2_LOAD_PATH [Done / 0 recs / 0.035s]"                                                    
[60] "06/12 15:55:50 DBG -- Inserted into I2B2_LOAD_TREE_FULL [Done / 1142 recs / 0.025s]"                                            
[61] "06/12 15:55:50 DBG -- Create i2b2 full tree [Done / 0 recs / 0.018s]"                                                           
[62] "06/12 15:55:51 DBG -- Create observation facts index [Done / 0 recs / 0.408s]"                                                  
[63] "06/12 15:55:51 DBG -- Create i2b2 full tree counts [Done / 0 recs / 0.024s]"                                                    
[64] "06/12 15:55:51 DBG -- Set c_visualattributes in i2b2 [Done / 209 recs / 0.030s]"                                                
[65] "06/12 15:55:51 DBG -- Finish fill in tree [Done / 0 recs / 0.005s]"                                                             
[66] "06/12 15:55:51 DBG -- Set sourcesystem_cd to null for added upper-level nodes [Done / 0 recs / 0.005s]"                         
[67] "06/12 15:55:51 DBG -- Delete counts for trial from I2B2DEMODATA concept_counts [Done / 209 recs / 0.006s]"                      
[68] "06/12 15:55:57 DBG -- Insert counts for trial into I2B2DEMODATA concept_counts [Done / 209 recs / 5.702s]"                      
[69] "06/12 15:55:57 DBG -- Nodes hidden with missing/zero counts for trial into I2B2DEMODATA concept_counts [Done / 0 recs / 0.006s]"
[70] "06/12 15:55:57 DBG -- Clean-up I2B2METADATA i2b2_secure [Done / 209 recs / 0.006s]"                                             
[71] "06/12 15:55:57 DBG -- Insert security data into I2B2METADATA i2b2_secure [Done / 209 recs / 0.013s]"                            
[72] "06/12 15:55:57 DBG -- End i2b2_load_clinical_data [Done / 0 recs / 0.005s]"                                                     
[73] "06/12 15:55:57 MSG Procedure completed successfully"                                                                            
[74] "06/12 15:55:57 MSG Job ID: 5084"                                                                                                
[75] "06/12 15:55:57 MSG Procedure completed successfully"                                                                            
[76] "06/12 15:55:57 MSG === FINISHED PROCESSING \\Inception ==="                                                                     
[77] "06/12 15:55:57 MSG ==== COMPLETED ===="                                                                                         

$stats
              File           Variable Variable Type   N null Mean Median IQR Min Max SD Count Required Validation rule QC missing data        QC data range
1  Comorbidity.txt            SUBJ_ID            ID 376    0   NA     NA  NA  NA  NA NA    NA      Yes              NA              OK                     
2  Comorbidity.txt        AE_ABDOPAIN          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
3  Comorbidity.txt          MH_ASTHMA          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
4  Comorbidity.txt      MH_DERMATITIS          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
5  Comorbidity.txt     MH_CHOLANGITIS          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
6  Comorbidity.txt            MH_COPD          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
7  Comorbidity.txt       MH_INFECTION          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
8  Comorbidity.txt         MH_SMOKING          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
9  Comorbidity.txt         MH_COELIAC          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
10 Comorbidity.txt             MH_CAD          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
11 Comorbidity.txt         MH_DIABETE          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
12 Comorbidity.txt        AE_DIARRHEA          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
13 Comorbidity.txt          MH_CANCER          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
14 Comorbidity.txt             MH_IBD          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
15 Comorbidity.txt     MH_HEREDANEMIA          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
16 Comorbidity.txt         MH_OBESITY          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
17 Comorbidity.txt             MH_PBC          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
18 Comorbidity.txt       MH_PSORIASIS          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
19 Comorbidity.txt          MH_STATIN          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
20 Comorbidity.txt         AE_STIPSIS          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
21 Comorbidity.txt     MH_THYROIDITIS          Text 376    0   NA     NA  NA  NA  NA NA    NA                       NA                                     
22  Medication.txt            SUBJ_ID            ID   0    0   NA     NA  NA  NA  NA NA    NA      Yes              NA              OK All values are empty
23  Medication.txt         CM_CORTICO          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
24  Medication.txt   CM_SULFASALAZINE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
25  Medication.txt      CM_CHOLOQUINE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
26  Medication.txt    CM_METHOTREXATE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
27  Medication.txt     CM_LEFLUNOMIDE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
28  Medication.txt    CM_AZATHIOPRINE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
29  Medication.txt    CM_MYCOPHENOLIC          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
30  Medication.txt    CM_CYCLOSPORINE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
31  Medication.txt CM_CYCLOPHOSPAMIDE          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
32  Medication.txt       CM_RITUXIMAB          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
33  Medication.txt            CM_ATNF          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
34  Medication.txt     CM_TOCILIZUMAB          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
35  Medication.txt       CM_ABATACEPT          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty
36  Medication.txt       CM_BELIMUMAB          Text   0    0   NA     NA  NA  NA  NA NA    NA                       NA                 All values are empty

If I then try to upload the 2nd file (Medication), it works:

$output
 [1] ">>> USING CONFIG: Config.groovy"                                                                                                
 [2] ">>> USING INTERACTIVE MODE"                                                                                                     
 [3] ">>> WILL STOP ON FAIL"                                                                                                          
 [4] ">>> OVERRIDING CONFIG: VISIT_NAME first"                                                                                        
 [5] ">>> FYI: using VISIT_NAME before DATA_VALUE as default behavior (per config or command line)"                                   
 [6] "06/12 16:01:26 MSG !!! TM_ETL VERSION 1.2.4-SNAPSHOT"                                                                           
 [7] "06/12 16:01:26 MSG ==== STARTED ===="                                                                                           
 [8] "06/12 16:01:26 MSG Using directory: ETL"                                                                                        
 [9] "06/12 16:01:26 MSG Processing directory: ETL"                                                                                   
[10] "06/12 16:01:26 MSG Processing directory: ETL/Inception"                                                                         
[11] "06/12 16:01:26 MSG === PROCESSING STUDIES IN '\\Inception' ==="                                                                 
[12] "06/12 16:01:26 MSG == Found study: Low Dimensional Data =="                                                                     
[13] "06/12 16:01:26 MSG Processing Clinical data"                                                                                    
[14] "06/12 16:01:26 MSG Connecting to database server"                                                                               
[15] "06/12 16:01:26 MSG Mapping file: AD2_Mapping_File.txt"                                                                          
[16] "06/12 16:01:27 MSG Processing data.txt"                                                                                         
[17] "06/12 16:01:27 MSG Processed 373 rows"                                                                                          
[18] "06/12 16:01:27 MSG Job ID: 5085"                                                                                                
[19] "06/12 16:01:27 MSG Run procedures: I2B2_LOAD_CLINICAL_DATA"                                                                     
[20] "06/12 16:01:27 MSG Study ID=AD2; Node=\\Inception\\Low Dimensional Data\\"                                                      
[21] "06/12 16:01:28 DBG -- Start i2b2_load_clinical_data for AD2 [Done / 0 recs / 0.000s]"                                           
[22] "06/12 16:01:28 DBG -- Load lt_src_clinical_data to work table [Done / 5222 recs / 0.056s]"                                      
[23] "06/12 16:01:28 DBG -- Set columns in wrk_clinical_data [Done / 5222 recs / 0.115s]"                                             
[24] "06/12 16:01:28 DBG -- Delete null data_values in wrk_clinical_data [Done / 0 recs / 0.006s]"                                    
[25] "06/12 16:01:28 DBG -- Remove pipes in data_value [Done / 0 recs / 0.006s]"                                                      
[26] "06/12 16:01:28 DBG -- Remove empty parentheses 1 [Done / 0 recs / 0.007s]"                                                      
[27] "06/12 16:01:28 DBG -- Remove empty parentheses 2 [Done / 0 recs / 0.007s]"                                                      
[28] "06/12 16:01:28 DBG -- Replace pipes with comma in data_label [Done / 0 recs / 0.006s]"                                          
[29] "06/12 16:01:28 DBG -- Set single visit_name to null [Done / 0 recs / 0.199s]"                                                   
[30] "06/12 16:01:28 DBG -- Remove tag markers [Done / 0 recs / 0.006s]"                                                              
[31] "06/12 16:01:28 DBG -- Set data_label to null when found in category_path [Done / 0 recs / 0.360s]"                              
[32] "06/12 16:01:28 DBG -- Remove leading, trailing, double spaces [Done / 5222 recs / 0.212s]"                                      
[33] "06/12 16:01:28 DBG -- Set visit_name to null if VISITNAME not in category_path [Done / 0 recs / 0.008s]"                        
[34] "06/12 16:01:28 DBG -- Set data_label to null if DATALABEL not in category_path [Done / 0 recs / 0.007s]"                        
[35] "06/12 16:01:28 DBG -- Set visit_name to null when found in data_label [Done / 0 recs / 0.008s]"                                 
[36] "06/12 16:01:28 DBG -- Set visit_name to null when found in data_value [Done / 0 recs / 0.008s]"                                 
[37] "06/12 16:01:28 DBG -- Insert numeric data into WZ wt_num_data_types [Done / 0 recs / 0.166s]"                                   
[38] "06/12 16:01:29 DBG -- Updated data_type flag for numeric data_types [Done / 0 recs / 0.006s]"                                   
[39] "06/12 16:01:29 DBG -- Add if missing DATALABEL, VISITNAME and DATAVALUE to category_path [Done / 5222 recs / 0.107s]"           
[40] "06/12 16:01:29 DBG -- Remove duplicates from wrk_clinical_data [Done / 0 recs / 0.012s]"                                        
[41] "06/12 16:01:29 DBG -- Check for duplicate key columns [Done / 0 recs / 0.011s]"                                                 
[42] "06/12 16:01:29 DBG -- Check for missing visit_names for subject_id/category/label/value  [Done / 1 recs / 0.011s]"              
[43] "06/12 16:01:29 DBG -- Create leaf nodes for trial [Done / 69 recs / 0.441s]"                                                    
[44] "06/12 16:01:29 DBG -- Updated last path component for timestamps [Done / 0 recs / 0.005s]"                                      
[45] "06/12 16:01:29 DBG -- Updated node name for leaf nodes [Done / 69 recs / 0.015s]"                                               
[46] "06/12 16:01:29 DBG -- Insert subject information into temp table [Done / 178 recs / 0.025s]"                                    
[47] "06/12 16:01:29 DBG -- Update subjects with changed demographics in patient_dimension [Done / 178 recs / 0.007s]"                
[48] "06/12 16:01:29 DBG -- Insert new subjects into patient_dimension [Done / 0 recs / 0.005s]"                                      
[49] "06/12 16:01:29 DBG -- Inserted new leaf nodes into I2B2DEMODATA concept_dimension [Done / 69 recs / 0.008s]"                    
[50] "06/12 16:01:29 DBG -- Updated name and data type in i2b2 if changed [Done / 0 recs / 0.006s]"                                   
[51] "06/12 16:01:29 DBG -- Inserted leaf nodes into I2B2METADATA i2b2 [Done / 69 recs / 0.017s]"                                     
[52] "06/12 16:01:29 DBG -- Delete old numerical fact records for append data [Done / 69 recs / 0.047s]"                              
[53] "06/12 16:01:33 DBG -- Collect observation facts [Done / 5222 recs / 3.417s]"                                                    
[54] "06/12 16:01:33 DBG -- Drop observation facts indexes [Done / 0 recs / 0.009s]"                                                  
[55] "06/12 16:01:33 DBG -- Insert trial into I2B2DEMODATA observation_fact [Done / 5222 recs / 0.083s]"                              
[56] "06/12 16:01:33 DBG -- Inserted into I2B2_LOAD_PATH [Done / 0 recs / 0.051s]"                                                    
[57] "06/12 16:01:33 DBG -- Inserted into I2B2_LOAD_TREE_FULL [Done / 1825 recs / 0.082s]"                                            
[58] "06/12 16:01:33 DBG -- Create i2b2 full tree [Done / 0 recs / 0.018s]"                                                           
[59] "06/12 16:01:33 DBG -- Create observation facts index [Done / 0 recs / 0.329s]"                                                  
[60] "06/12 16:01:33 DBG -- Create i2b2 full tree counts [Done / 0 recs / 0.028s]"                                                    
[61] "06/12 16:01:33 DBG -- Set c_visualattributes in i2b2 [Done / 335 recs / 0.041s]"                                                
[62] "06/12 16:01:33 DBG -- Finish fill in tree [Done / 0 recs / 0.005s]"                                                             
[63] "06/12 16:01:33 DBG -- Set sourcesystem_cd to null for added upper-level nodes [Done / 0 recs / 0.005s]"                         
[64] "06/12 16:01:33 DBG -- Delete counts for trial from I2B2DEMODATA concept_counts [Done / 209 recs / 0.006s]"                      
[65] "06/12 16:01:43 DBG -- Insert counts for trial into I2B2DEMODATA concept_counts [Done / 335 recs / 9.927s]"                      
[66] "06/12 16:01:43 DBG -- Nodes hidden with missing/zero counts for trial into I2B2DEMODATA concept_counts [Done / 0 recs / 0.006s]"
[67] "06/12 16:01:43 DBG -- Clean-up I2B2METADATA i2b2_secure [Done / 209 recs / 0.006s]"                                             
[68] "06/12 16:01:43 DBG -- Insert security data into I2B2METADATA i2b2_secure [Done / 335 recs / 0.016s]"                            
[69] "06/12 16:01:43 DBG -- End i2b2_load_clinical_data [Done / 0 recs / 0.005s]"                                                     
[70] "06/12 16:01:43 MSG Procedure completed successfully"                                                                            
[71] "06/12 16:01:43 MSG Job ID: 5086"                                                                                                
[72] "06/12 16:01:43 MSG Procedure completed successfully"                                                                            
[73] "06/12 16:01:43 MSG === FINISHED PROCESSING \\Inception ==="                                                                     
[74] "06/12 16:01:43 MSG ==== COMPLETED ===="                                                                                         

$stats
       File           Variable Variable Type   N null Mean Median IQR Min Max SD Count Required Validation rule QC missing data QC data range
1  data.txt            SUBJ_ID            ID 373    0   NA     NA  NA  NA  NA NA    NA      Yes              NA              OK            NA
2  data.txt         CM_CORTICO          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
3  data.txt   CM_SULFASALAZINE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
4  data.txt      CM_CHOLOQUINE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
5  data.txt    CM_METHOTREXATE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
6  data.txt     CM_LEFLUNOMIDE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
7  data.txt    CM_AZATHIOPRINE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
8  data.txt    CM_MYCOPHENOLIC          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
9  data.txt    CM_CYCLOSPORINE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
10 data.txt CM_CYCLOPHOSPAMIDE          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
11 data.txt       CM_RITUXIMAB          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
12 data.txt            CM_ATNF          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
13 data.txt     CM_TOCILIZUMAB          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
14 data.txt       CM_ABATACEPT          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
15 data.txt       CM_BELIMUMAB          Text 373    0   NA     NA  NA  NA  NA NA    NA                       NA                            NA
xuatiknc commented 7 years ago

Hi,

could you please post your AD2_Mapping_File.txt file as well as a few first lines of files Comorbidity.txt, Medication.txt, and data.txt?

kforner commented 7 years ago

Sorry, was my mistake. When uploading the 2 tables at once, the second one did not have the STUDY_ID variable defined.