Batch inserts fail due to always using original schema

z3z1ma / target-bigquery

target-bigquery is a Singer target for BigQuery. It supports storage write, GCS, streaming, and batch load methods. Built with the Meltano SDK.

MIT License

28 stars 38 forks source link

Batch inserts fail due to always using original schema #105

Open heynspaul opened 1 month ago

heynspaul commented 1 month ago

When running a batch job, batch_job.py, job_config variable always assumes raw schema from source. No transformation is applied. This means that http 400 is thrown due to chars not allowed when attempting to insert.

Suggested change: always switch on transforms for in the job_config method/property when denormalized mode: