xitongsys / parquet-go

pure golang library for reading/writing parquet file
Apache License 2.0
1.27k stars 293 forks source link

Reader/writer parallel settings - provide more information on what they do #338

Closed mdgraboski closed 3 years ago

mdgraboski commented 3 years ago

The readme docs say only this about the parallel / goroutine settings for readers and writers:

Read/Write initial functions have a parallel parameters np which is the number of goroutines in reading/writing.

What does this np parameter actually do? How should I adjust it? Should I set it to NumCPU on my system? Does it write multiple output files, or does it just parallelize the process of creating the output files? One or two sentences about what these settings actually do, and how to tune them, would be helpful.

xitongsys commented 3 years ago

hi, @mdgraboski improved PR. Hope it is helpful.