ghdna / athena-express

Athena-Express can simplify executing SQL queries in Amazon Athena AND fetching cleaned-up JSON results in the same synchronous or asynchronous request - well suited for web applications.
https://www.npmjs.com/package/athena-express
MIT License
179 stars 70 forks source link

creating queryIterable method #66

Open Farenheith opened 2 years ago

Farenheith commented 2 years ago

The added method aims to offer a seamless way to obtain paginated results from Athena through the async iterable contract.

It abstracts the nextToken manipulation making the call for paginated results much simpler, like in the example below:

async function main() {
    const results = athenaExpress.queryIterable("SELECT * from students LIMIT 100", 10);
    for await (const page of results) {
          console.log(page.Items);
    }
}
main();

The default pagination value (the method second parameter) is 999, because 1000 are throwing an error due to the paginationFactor treatment in the getQueryResultsFromS3 method.