search

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.35k stars 1.69k forks source link

Not hit datacache while reading the Paimon table #36943

Open taoshengyijiua opened 7 months ago

taoshengyijiua commented 7 months ago

Enhancement

1702449611096_329B4303-E688-42c6-B1A6-C839A098E81A When reading the Paimon external table through StarRocks, enabling datacache is invalid. It is recommended to add this feature to improve the efficiency of reading the Paimon external table

miomiocat commented 7 months ago

Thank you for the feedback.
Data cache does not yet support Paimon, but we will evaluate the addition of this feature as soon as possible.

lmagic233 commented 4 months ago

@miomiocat I've tested reading Paimon tables on S3 just before, with identical aggregation queries, and both enable_scan_datacache and enable_populate_datacache parameters are set to true. The query profile shows that data cache is effective. Is this a false indicator or it's truely available? Thanks.

image
Pandas886 commented 2 months ago

Does piamon now support data cache?

miomiocat commented 2 months ago

@miomiocat I've tested reading Paimon tables on S3 just before, with identical aggregation queries, and both enable_scan_datacache and enable_populate_datacache parameters are set to true. The query profile shows that data cache is effective. Is this a false indicator or it's truely available? Thanks. image

paimon datacache available from 3.2.3 version

miomiocat commented 2 months ago

Does piamon now support data cache?

yes. available from 3.2.3 version