DATA-DOG / go-sqlmock

Sql mock driver for golang to test database interactions
Other
5.95k stars 404 forks source link

Add FromCsvReader method to allow building rows with customized csv reader. #339

Open stayingalivee opened 2 weeks ago

stayingalivee commented 2 weeks ago

Proposal

Under the existing design, user has to manually modify their csv string to match with csv reader defaults. func (r *Rows) FromCSVString(s string) *Rows function initialize a csv reader with defaults. The user may have a pipe separated csv or some other token. Also, the user cannot enable lazyQoutes (see: https://cs.opensource.google/go/go/+/refs/tags/go1.22.4:src/encoding/csv/reader.go;l=136) which is sometimes needed if you have a json string in the csv file without having to modify the csv extensively to match the defaults.

My proposal is to allow the user to define their csv reader freely with their own custom options. Easiest way to do it without breaking existing code is the following:

func (r *Rows) FromCSVReader(csvReader *csv.Reader) *Rows {
    for {
        res, err := csvReader.Read()
        if err != nil {
            if errors.Is(err, io.EOF) {
                break
            }
            panic(fmt.Sprintf("Parsing CSV string failed: %s", err.Error()))
        }

        row := make([]driver.Value, len(r.cols))
        for i, v := range res {
            row[i] = CSVColumnParser(strings.TrimSpace(v))
        }
        r.rows = append(r.rows, row)
    }
    return r
}

you can also check this commit that I pushed on my fork: https://github.com/stayingalivee/go-sqlmock/commit/d0736ed6b8a4052c2198ebf41cae30063e0c3d35

Use-cases

I added a test to demonstrate the use-case. see: https://github.com/stayingalivee/go-sqlmock/commit/d0736ed6b8a4052c2198ebf41cae30063e0c3d35#diff-335b77abfb8789fd51a279c7cbdbdae90b384df153ce1bbac40aa1468d5bbb89R472

for ease of access I will paste the test below:

func TestCSVRowParserFromCsvParser(t *testing.T) {
    t.Parallel()

    line := "a|NULL|[\"a\",\"b\"]"

    r := strings.NewReader(line)
    csvReader := csv.NewReader(r)
    csvReader.Comma = '|'
    csvReader.LazyQuotes = true

    rs := NewRows([]string{"col1", "col2", "col3"}).FromCSVReader(csvReader)
    db, mock, err := New()
    if err != nil {
        t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
    }
    defer db.Close()

    mock.ExpectQuery("SELECT").WillReturnRows(rs)

    rw, err := db.Query("SELECT")
    if err != nil {
        t.Fatalf("unexpected error: %s", err)
    }
    defer rw.Close()
    var col1 string
    var col2 []byte
    var col3 string
    var col3Json []string

    rw.Next()
    if err = rw.Scan(&col1, &col2, &col3); err != nil {
        t.Fatalf("unexpected error: %s", err)
    }
    if col1 != "a" {
        t.Fatalf("expected col1 to be 'a', but got [%T]:%+v", col1, col1)
    }
    if col2 != nil {
        t.Fatalf("expected col2 to be nil, but got [%T]:%+v", col2, col2)
    }

    err = json.Unmarshal([]byte(col3), &col3Json)
    if err != nil {
        t.Fatalf("expected valid json, but got [%T]:%+v", col3, col3)
    }

    if !reflect.DeepEqual(col3Json, []string{"a", "b"}) {
        t.Fatalf("expected col3Json to be [\"a\", \"b\"], but got [%T]:%+v", col3Json, col3Json)
    }
}

If you find this helpful let me know so I can submit a pull request.