turbot / steampipe

Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
https://steampipe.io
GNU Affero General Public License v3.0
6.84k stars 264 forks source link

Push down support for `order by` statements #4098

Closed Joeturbot closed 4 months ago

Joeturbot commented 7 months ago

Is your feature request related to a problem? Please describe. I have a query using the Guardrails plugin that performs too slowly when using the order by clause. This query runs very quickly.

select id, process_id, notification_type, create_timestamp, actor_identity_id, resource_type_uri, resource_akas
from guardrails_notification
where filter = 'notificationType:resource'
limit 10;

while this one takes forever:

select id, process_id, notification_type, create_timestamp, actor_identity_id, resource_type_uri, resource_akas
from guardrails_notification
where filter = 'notificationType:resource'
order by create_timestamp desc
limit 10;

The difference is the inclusion of order by create_timestamp desc.

Describe the solution you'd like Add the ability to "push down" an order by clause to the plugin. Otherwise, all rows must be retrieved first then limited to 10. The Guardrails notification table can easily contain millions of rows.

Describe alternatives you've considered Filtering on the Guardrails-side provides the required data at the same speeds as the query above without the order by clause in it.

select id, process_id, notification_type, create_timestamp, actor_identity_id, resource_type_uri, resource_akas
from guardrails_notification
where filter = 'notificationType:resource sort:-createTimestamp'
limit 10;

Additional context

github-actions[bot] commented 5 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] commented 4 months ago

This issue was closed because it has been stalled for 90 days with no activity.