search

ghdna / athena-express

Athena-Express can simplify executing SQL queries in Amazon Athena AND fetching cleaned-up JSON results in the same synchronous or asynchronous request - well suited for web applications.
https://www.npmjs.com/package/athena-express
MIT License
179 stars 70 forks source link

Is there a way to get stats, but not the results? #53

Closed mcbergsma closed 3 years ago

mcbergsma commented 3 years ago

I need to run a few fairly lengthy queries so I've created an endpoint that triggers the query to run.

I run it with a config like { skipResults: true, waitForResults: false } and get back the QueryExecutionId.

I'd like to store the QueryExecutionID and then be able to basically run athenaExpress(queryExecutionId) every N seconds with a config along the lines of 

{
  getStats: true,
  skipResults: true
}

So I can get the S3Location back from getStats and do what I need to do with it (rename, presign etc.) But I haven't figured out a way to just get the S3Location without it also trying to send back all the data (which is a few hundred MB in this case.

I'm sure I can use the QueryExecutionId along with the AWS SDK to make it work, but wanted to make sure I'm not missing something already existing in athena-express.

ghdna commented 3 years ago

sorry for a delayed response. Did you figure this out? You can use {waitForResults: false} for your usecase. Details in readme