Unlike csv, parquet stores schema info as well. Bigquery changes date columns db dtype column. This can cause inconsistencies for downstream operations like writing to pandas df.
Going forward, we may need to check if there are any date columns in upcoming indices (pathology, clinical, etc)
Unlike csv, parquet stores schema info as well. Bigquery changes date columns db dtype column. This can cause inconsistencies for downstream operations like writing to pandas df.
Going forward, we may need to check if there are any date columns in upcoming indices (pathology, clinical, etc)