apache / drill

Apache Drill is a distributed MPP query layer for self describing data
https://drill.apache.org/
Apache License 2.0
1.93k stars 984 forks source link

Apache Drill Query Execution PLAN Doesn't use mongo db index #2826

Closed dwevedivaibhav closed 3 months ago

dwevedivaibhav commented 11 months ago

Hi Team,

I am trying to execute the millions of record query from mongo db storage with timestamp filter, it getting slow even though i created the index of timestamp in mongo collection, but its taking so long time to execute and some time time its failing also due to huge record.

Please find the sample query which i am executing from apache drill

select * FROM mongo.sampletable WHERE SentTime >= TO_TIMESTAMP('2023-08-10 00:00:00', 'yyyy-MM-dd HH:mm:ss') AND SentTime < TO_TIMESTAMP('2023-08-17 00:00:00', 'yyyy-MM-dd HH:mm:ss') LIMIT 10

jnturton commented 11 months ago

I wonder if your constant timestamp expressions are being folded by the planner. Do you get the same performance from the next query?

SELECT * FROM mongo.sampletable
WHERE SentTime >= TIMESTAMP '2023-08-10 00:00:00'
AND SentTime < TIMESTAMP '2023-08-17 00:00:00'
LIMIT 10
dwevedivaibhav commented 3 months ago

Yes Same problem @jnturton

jnturton commented 3 months ago

Duplicate of #2906.