Incorrect data-types in schema.json

cloudflare / cloudflare-gcp

Google Cloud Function to push json files from GC Storage to Big Query

Apache License 2.0

84 stars 49 forks source link

Incorrect data-types in schema.json #38

Closed neturbo closed 4 years ago

neturbo commented 4 years ago

Hi,

I found the following data-types might course errors during data migration.

https://github.com/cloudflare/cloudflare-gcp/blob/master/logpush-to-bigquery/schema.json Line#123: "name": "EdgeEndTimestamp", Line#124: "type": "TIMESTAMP" (INTEGER)

Line#183: "name": "EdgeStartTimestamp", Line#184: "type": "TIMESTAMP" (INTEGER)

Line#258: "name": "WAFFlags", Line#259: "type": "INTEGER" (STRING)

My reference from: https://developers.cloudflare.com/logs/log-fields/

Thank you for your time.

shagamemnon commented 4 years ago

Thanks @neturbo. Have you confirmed that this causes any issues with the Cloud Storage to Big Query integration? If not, I think its best to close this issue on this repo as our goal here it proper formatting in BigQuery -- regardless of the data types listed in the official schema.