Open dtaniwaki opened 1 month ago
@lucydodo @justinclift Could you consider merging this feature?
It looks good, but since the version of the pyathena
package goes up quite a bit,
so we should provbably look into the possibility of other side effectes before merging.
What type of PR is this?
Description
Currently, redash makes a query everytime the query page opens and it costs a lot if the query is too heavy.
On the other hand, there is a feature that efficiently uses query results in Athena. https://docs.aws.amazon.com/athena/latest/ug/reusing-query-results.html
pyathena (>=2.18.0) supports the option. https://github.com/laughingman7743/PyAthena/blob/6b8f0e94abd09115ba1277616ca84372e2f15d56/pyathena/common.py#L112-L113 It seems the reason of updating the major version is dropping python versions. and as far as I tried, queries work fine.
We may want to turn on/off the option per query but it will break the compatibility with the other data sources, so I made it a data source option. Could you consider adding this feature?
How is this tested?
Run an Athena query with this option with manually container image.
Related Tickets & Documents
N/A
Mobile & Desktop Screenshots/Recordings (if there are UI changes)