firebase / extensions

Source code for official Firebase extensions
https://firebase.google.com/products/extensions
Apache License 2.0
888 stars 375 forks source link

firebase-bigquery-export:schema-view Timestamps filtered out on "latest" view #347

Closed neilpoulin closed 4 years ago

neilpoulin commented 4 years ago

[REQUIRED] Step 2: Describe your configuration

[REQUIRED] Step 3: Describe the problem

When generating a schema for the firebase bigquery export extension, Firebase Timestamp values do not show up in the latest schema but do in the changelog.

Looking in the code, there is a message that claims TIMESTAMP values can not be grouped, and are thus excluded. I feel like this may be a mistake. Here's the sample code in firestore-bigquery-export/scripts/gen-schema-view/src/schema.ts,

   // For "latest" data views, certain types of fields cannot be used in
    // "GROUP BY" clauses. We keep track of them so they can be explicitly
    // transformed into groupable types later.
    if (field.type === "array") {
      arrays.push(qualifyFieldName(prefix, field.name));
    }
    if (field.type === "geopoint") {
      geopoints.push(qualifyFieldName(prefix, field.name));
    }

     if (field.type === "timestamp") {
       timestamps.push(qualifyFieldName(prefix, field.name));
     }

I don't understand why this applies to timestamps (it's very possible I simply am ignorant on the subject -- I'm no SQL expert). When I comment out the timestamp clause, the schemas and resulting views work exactly as expected, with SQL Timestamp values where I had registered a timestamp in my schema.

Steps to reproduce:

My "fix" Simply remove the if statement schema.ts that adds timestamp values to the exclusion array.

russellwheatley commented 4 years ago

Hey @Neilpoulin, thanks for raising the issue. You're correct, I'll be making a PR that will allow the timestamp to come through in the appropriate column that was mapped in the schema for the "latest" View.