m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.
Apache License 2.0
13 stars 5 forks source link

BQ load from GCS encountering errors #291

Open gfr10598 opened 4 years ago

gfr10598 commented 4 years ago

We occasionally see errors like this:

Location: "gs://json-mlab-sandbox/ndt/ndt7/2020/06/17/20200617T220807.881508Z-ndt7-mlab2-hnd02-ndt-.json"; Message: "Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details."; Reason: "invalid"

They may be persistent, and we may need to have a way to work around them - possibly by deleting the json files from GCS and repeating the load.