Closed imalkov82 closed 2 years ago
Use NewGcsFileReader to read:
gcsReader, err := gcs.NewGcsFileReader(context.Background(), projectId, bucket, objectName))
if err != nil {
return nil, fmt.Errorf("failed to open GCS object [%s]: %s", uri, err.Error())
}
reader, err := reader.NewParquetReader(gcsReader, nil, 4)
// everything else is same as dealing with local file.
And NewGcsFileWriter to write.
Assume you have environment configured already to access GCS, if not: https://cloud.google.com/docs/authentication/production#automatically
Awesome, thank you @hangxie !
Can you provide an example of how to use the package to write/read parquet files to Google Cloud Storage