Unable to unmarshal first column as string

[scharph]

I have the following csv data

date;municipality_id;municipality_name;municipality_population;dose_1;dose_2;dose_3;valid_certificates;valid_certificates_percent
2021-12-03T23:59:59+01:00;10101;Eisenstadt;14895;11420;10568;4145;10924;73.340047

i want to unmarshal it to the following struct

type Vacc struct {
    Date                   string     `json:"date" csv:"date"`
    MunicipalityId         int32      `json:"municipality_id" csv:"municipality_id"`
    MunicipalityName       string     `json:"municipality_name" csv:"municipality_name"`
    MunicipalityPopulation int        `json:"municipality_population" csv:"municipality_population"`
    Dose1                  int        `json:"dose_1" csv:"dose_1"`
    Dose2                  int        `json:"dose_2" csv:"dose_2"`
    Dose3                  int        `json:"dose_3" csv:"dose_3"`
    ValidCertificates      int32      `json:"valid_certificates" csv:"valid_certificates"`
    ValidCertificatesPerc  float32    `json:"valid_certificates_percent" csv:"valid_certificates_percent"`
}

Why is csvutil not able to parse the first column "date" as string?

Full Code:

csvReader := csv.NewReader(resp.Body)
csvReader.Comma = ';'

dec, err := csvutil.NewDecoder(csvReader)
if err != nil {
    log.Fatal(err)
}

var vaccs []Vacc
for {
    var v Vacc
    if err := dec.Decode(&v); err == io.EOF {
        break
    } else if err != nil {
        log.Fatal(err)
    }
    vaccs = append(vaccs, v)
}
log.Printf("%+v\n", vaccs[0])

[jszwec]

Why is csvutil not able to parse the first column "date" as string?

it is able to decode it (csvutil is not a parser, csv.Reader is): https://go.dev/play/p/ba5bY63qYs9

the error that you are getting would be useful. It is likely that you have some hidden characters in your byte stream

[scharph]

I load the csv from web and I don't get errors

go 17.2

here is the actual code ...

package main

import (
    "encoding/csv"
    "log"
    "net/http"

    "github.com/davecgh/go-spew/spew"
    "github.com/jszwec/csvutil"
)

const (
    URL = "https://info.gesundheitsministerium.gv.at/data/COVID19_vaccination_municipalities.csv"

// URL = "https://gist.githubusercontent.com/scharph/4fa350a71d4f4be8d6de2159fc90f92a/raw/7984a74a6f7165aeae7a354186404335320b472a/vaccination_municipalities.csv"
)

type Vacc struct {
    Date                   string  `json:"date" csv:"date"`
    MunicipalityId         int32   `json:"municipality_id" csv:"municipality_id"`
    MunicipalityName       string  `json:"municipality_name" csv:"municipality_name"`
    MunicipalityPopulation int     `json:"municipality_population" csv:"municipality_population"`
    Dose1                  int     `json:"dose_1" csv:"dose_1"`
    Dose2                  int     `json:"dose_2" csv:"dose_2"`
    Dose3                  int     `json:"dose_3" csv:"dose_3"`
    ValidCertificates      int32   `json:"valid_certificates" csv:"valid_certificates"`
    ValidCertificatesPerc  float32 `json:"valid_certificates_percent" csv:"valid_certificates_percent"`
}

func main() {
    resp, err := http.Get(URL)
    if err != nil {
        log.Fatal(err)
    }

    defer resp.Body.Close()
    csvReader := csv.NewReader(resp.Body)
    csvReader.Comma = ';'

    dec, err := csvutil.NewDecoder(csvReader)
    if err != nil {
        log.Fatal(err)
    }

    var vaccs []Vacc
    if err := dec.Decode(&vaccs); err != nil {
        log.Fatal(err)
    }
    spew.Dump(vaccs)
}

Ok I think there's something wrong with my data source url

[jszwec]

your data starts with BOM -> (EF BB BF), you need to drop it, and it's gonna work. It's related to: https://github.com/jszwec/csvutil/issues/32

[scharph]

got it .. thanks a lot