xitongsys / parquet-go

pure golang library for reading/writing parquet file
Apache License 2.0
1.27k stars 293 forks source link

Add support for batch read #554

Open tchaton opened 1 year ago

tchaton commented 1 year ago

Hey there,

Thanks a lot for this library !

I wondered if it could be possible to have a batch read where we could provide the list of indices to read directly. I believe this would be faster than using a combinaison of SkipRows and Reads.

    num = int(pr.GetNumRows())
    for i := 0; i < num/10; i++ {
        stus := make([]Student2, 10) //read 10 rows
        if err = pr.Read(&stus, [0, 2, 4, 6, 8, ..., 18]); err != nil {
            log.Println("Read error", err)
        }
        log.Println(stus)
    }

Best, T.C