search

vapor-community / postgresql

Robust PostgreSQL interface for Swift
MIT License
131 stars 33 forks source link

Lazily pull results from DB #73

Closed juri closed 3 years ago

juri commented 6 years ago

This PR adds support for lazily loading results of a DB query. That is: when you execute a query, instead of getting back a node that contains an array of results, you get a sequence of nodes that is fetched on demand as you iterate through them.

I have very little experience using libpq, so any insight from people more knowledgeable than me is welcome.

Background

The in-memory array of all the result rows execute returns is not great if you have a huge number of rows you want to process one by one. That behaviour probably can't be changed, but this PR adds another method that works lazily.

Interface

Adding a version of execute with a different return type would have been possible, but it would have broken existing code because it would in some cases require extra annotation at the call site. So instead I added a new method, pullExecute. Any better suggestions are welcome.

The return type from pullExecute is ResultNodeSequence. It might have been possible to shoehorn the functionality into Result, but keeping it separate felt simpler.

Breaking changes

I'm introducing one API change: Bind.result is now typed as Any? instead of Result?. I don't know if that is a problem. The tests still pass, but it doesn't mean some other code doesn't depend on that type, as it (like everything else in this library, for some reason) is public. If this change is problematic, I'll have to go back to thinking about adding this stuff into Result.

Another thing that won't break compilation but may break other assumptions somewhere is that after you call pullExecute, you shouldn't use the same connection for anything else before closing the returned ResultNodeSequence, either by releasing it or by calling close on it.

natebird commented 6 years ago

You may want to look at vapor/postgres because that will be the officially support postgres driver for Vapor 3+.

Joannis commented 6 years ago

Vapor/postgres is developed for asynchronous queries so cannot do this. For synchronous APIs I love this pattern. I've been using that in MongoKitten for about 2 years now and it feels really nice