GoogleCloudDataproc / flink-bigquery-connector

BigQuery integration to Apache Flink's Table API
Apache License 2.0
15 stars 11 forks source link

Improve debugability #164

Open prashastia opened 3 days ago

prashastia commented 3 days ago
  1. For Datastream API, in case the source records' schema does not match the expected BQ Table(sink) schema, the deserializer (BigQueryDeserializationSchema) is unable to forward the mismatched record further to the pipeline. Hence, making this error more verbose to pinpoint the underlying cause instead of ClassCastException currently being seen due to the mismatch.

  2. Fix error in writing time type to BigQuery. LocalDateTime dropping 00 seconds value when parsing date string value with 00 seconds like "2018-07-06 00:00:00". Parsing now using ISO format to ensure a 00 at the end. /gcbrun