goccy / bigquery-emulator

BigQuery emulator server implemented in Go
MIT License
743 stars 86 forks source link

Unmarshal fails to a struct with time.Time type when using the cloud.google.com/go/bigquery library #275

Closed VDVsx closed 1 month ago

VDVsx commented 3 months ago

What happened?

Error occurs when trying to query records using the Go library(cloud.google.com/go/bigquery v1.59.1) previously saved using the same lib, that contain a time.Time type.

strconv.ParseInt: parsing "1709294011.271962": invalid syntax

What did you expect to happen?

The records can be loaded as before.

How can we reproduce it (as minimally and precisely as possible)?

Emulator running with project/dataset/table in place. Make sure cloud.google.com/go/bigquery > v.159.0.

Simple example(sorry that the formatting is a bit off after c&p)

import (
       "context"
    ....

    "cloud.google.com/go/bigquery"
)

type TestTs struct {
    Name                         string `bigquery:"name"`
    ReportTime                   time.Time `bigquery:"report_time"`
}

func testSaveTimestamp() error {
  ctx := context.Background()
  projectID := "projectID"
  datasetID := "testDataset"
  tableID := "testTable"

  bqClient, err := bigquery.NewClient(ctx,...) // add other connection parameters
  if err != nil {
      return err
  }

  // Insert data
  testData := []TestTs{
  {
    Name:               "test1",
    ReportTime:         time.Now().UTC(),
 },
 {
       Name:               "test2",
       ReportTime:         time.Now().UTC(),
 },
 }

  u := bqClient.Dataset(datasetID).Table(tableID).Inserter()
  err = u.Put(ctx, testData)
  if err != nil {
      return fmt.Errorf("failed to insert rows: %w", err)
  }

  // Load the data
  it := bqClient.Dataset(datasetID).Table(tableID).Read(ctx)
  var tData []TestTs
  for {
      var ts TestTs
      err := it.Next(&ts)
      if err == iterator.Done {
          break
      }
      if err != nil {
          return err // returns here the mentioned error
      }
      tData = append(tData, ts)
  }
}

Anything else we need to know?

This is likely related to this commit in the Go lib, since it started to happen after this release: https://github.com/googleapis/google-cloud-go/pull/9368

arhea commented 3 months ago

I have also seen this issue. I will also add, I tried changing the type to bigquery.NullTimestamp and have the same problem.

ohaibbq commented 3 months ago

Thanks for the test case and the wonderful report. @goccy @totem3 I've opened a PR here #276

arhea commented 2 months ago

Downgrading to Go SDK v1.58.0 is working as a temporary workaround.

arhea commented 2 months ago

Confirmed in release notes: https://github.com/googleapis/google-cloud-go/releases/tag/bigquery%2Fv1.59.0

arhea commented 1 month ago

Thanks @goccy and @ohaibbq!