segmentio / parquet-go

Go library to read/write Parquet files
https://pkg.go.dev/github.com/segmentio/parquet-go
Apache License 2.0
341 stars 102 forks source link

panic: cannot create parquet value of type DOUBLE from go value of type [][]float64 #473

Open yonesko opened 1 year ago

yonesko commented 1 year ago

Hello! I have json row:

{
  "geo_geo_bbox": {
    "type": "Polygon",
    "coordinates": [
      [
        [
          -79.76259,
          40.477383
        ],
        [
          -79.76259,
          45.015851
        ],
        [
          -71.777492,
          45.015851
        ],
        [
          -71.777492,
          40.477383
        ],
        [
          -79.76259,
          40.477383
        ]
      ]
    ]
  }
}

Which I try to write into Parquet. I generated golang struct:

&struct { 
    Geo_geo_bbox struct { 
        Type string "parquet:\"type,optional,plain\" json:\"type\" csv:\"type\""; 
        Coordinates [][][]float64 "parquet:\"coordinates,optional,list\" json:\"coordinates\" csv:\"coordinates\"" 
    } "parquet:\"geo_geo_bbox,optional\" json:\"geo_geo_bbox\" csv:\"geo_geo_bbox\"" 
}
{
    Type:"Polygon", 
    Coordinates:[][][]float64 {
        [][]float64{
            []float64{-79.76259, 40.477383}, 
            []float64{-79.76259, 45.015851}, 
            []float64{-71.777492, 45.015851}, 
            []float64{-71.777492, 40.477383}, 
            []float64{-79.76259, 40.477383}
        }
    }
}
}

and your lib created a Shema:

message SomeMsg {
    optional group geo_geo_bbox {
        optional binary type (STRING);
        optional group coordinates (LIST) {
            repeated group list {
                repeated double element;
            }
        }
    }
}

and I get error during parquetWriter.Write(row):

panic: cannot create parquet value of type DOUBLE from go value of type [][]float64

Any ideas ?