Open qianheng-aws opened 3 months ago
Is your feature request related to a problem? Sort or Filter cannot be pushed down into TableScan due to Project.
POST _plugins/_ppl/_explain { "query": """ source=opensearch_dashboards_sample_data_flights | fields FlightTimeMin |where FlightTimeMin > 0 | sort FlightTimeMin """ } # response { "root": { "name": "ProjectOperator", "description": { "fields": "[FlightTimeMin]" }, "children": [ { "name": "SortOperator", "description": { "sortList": { "FlightTimeMin": { "sortOrder": "ASC", "nullOrder": "NULL_FIRST" } } }, "children": [ { "name": "FilterOperator", "description": { "conditions": ">(FlightTimeMin, 0)" }, "children": [ { "name": "ProjectOperator", "description": { "fields": "[FlightTimeMin]" }, "children": [ { "name": "OpenSearchIndexScan", "description": { "request": """OpenSearchQueryRequest(indexName=opensearch_dashboards_sample_data_flights, sourceBuilder={"from":0,"size":200,"timeout":"1m","_source":{"includes":["FlightTimeMin"],"excludes":[]}}, searchDone=false)""" }, "children": [] } ] } ] } ] } ] } }
What solution would you like? Option1: remove ProjectOperator after pushed it down into TableScan. Not clear of the background why we intentionally wants to keep it here. https://github.com/opensearch-project/sql/blob/14a80a95fb5fa36781b46f28ba52d406927e21c0/opensearch/src/main/java/org/opensearch/sql/opensearch/storage/scan/OpenSearchIndexScanQueryBuilder.java#L79-L83
Option2: support push down Sort/Filter under Project.
What alternatives have you considered?
Do you have any additional context? Add any other context or screenshots about the feature request here.
[Catch All Triage - 1, 2, 3, 4, 5]
Is your feature request related to a problem? Sort or Filter cannot be pushed down into TableScan due to Project.
What solution would you like? Option1: remove ProjectOperator after pushed it down into TableScan. Not clear of the background why we intentionally wants to keep it here. https://github.com/opensearch-project/sql/blob/14a80a95fb5fa36781b46f28ba52d406927e21c0/opensearch/src/main/java/org/opensearch/sql/opensearch/storage/scan/OpenSearchIndexScanQueryBuilder.java#L79-L83
Option2: support push down Sort/Filter under Project.
What alternatives have you considered?
Do you have any additional context? Add any other context or screenshots about the feature request here.