Open morningman opened 1 year ago
I'm interested in this feature. Could you assign it to me? Thank you
I'm interested in this feature. Could you assign it to me? Thank you
Hello, wanna get to know is there any progress about this feature?
Hi, I'm ready to submit it now.
Search before asking
Description
Currently, if user modify the file on hdfs directly, no through hive. The changes of file will not be noticed by Doris and user will get wrong data.
One approach is to refresh the catalog at fixed rate. But this will have to refresh all catalog and invalidate all caches.
So I suggest to set TTL(Time-to-Live) config of File Cache, so that the stale file info will be invalidated automatically after expiring.
Use case
And file cache of all tables in this catalog will be expired after 60 seconds. You can also set
file_meta_cache_ttl_second
to 0 to disable file cache.Related issues
No response
Are you willing to submit PR?
Code of Conduct