Discussion: useful to return rows early using iter.Seq?

[rorycl]

Thanks, @jackc, for the wonderful pgx ecosystem.

Feature request (not related to a problem) Might it be useful to consider using an iter.Seq (scheduled for inclusion in go 1.23 after the current rangefunc experiments) version of, for example, pgx.CollectRows? This could yield results early so that the client could for example start writing output early or stop processing the results. This might also save collecting all the results in memory.

Describe the solution you'd like Something like pgx.CollectRowsIter[T any](rows Rows, fn RowToFunc[T]) (iter.Seq[T]) might be useful.

Describe alternatives you've considered One could always make one's own iterator off the results, although that means reading all of the results into memory. Eg https://github.com/rorycl/iterfolder/blob/1f3fc161ab3efeed4a00a71a448b2b83a59e30be/example/dbCompoundExample.go#L123

Additional context None

[jackc]

We may want to support the new Go 1.23 range iterator at some point. But it doesn't provide any new functionality with regard to streaming. Streaming results is actually how the lower level Rows.Next and Rows.Scan work.

The point of CollectRows is to buffer the entire results before processing so you don't have to handle errors mid result processing.

You can also use https://pkg.go.dev/github.com/jackc/pgx/v5#ForEachRow for a slightly higher level interface around Rows.Next and Rows.Scan.

[rorycl]

Thanks very much for your comment. I had completely overlooked Rows.Next etc.

I was able to use ForEachRow to produce an iter.Seq, although I'm not sure how to unwind the nested functions safely. Perhaps I should use iter.Seq Pull instead. See https://github.com/rorycl/iterfolder/blob/f9a36085d36971afd258050b3f787f05340e4135/example/dbCompoundExample2.go#L119

Anyhow, thanks for commenting. Buffering the entire results set before processing makes sense. Please feel free to close this issue.