StarRocks / starrocks-connector-for-apache-flink

Apache License 2.0
192 stars 154 forks source link

[Enhancement] look function support partial cache #293

Open Jin-H opened 11 months ago

Jin-H commented 11 months ago

I found that the current connector supports the lookup function, but the implementation is to fully scan the table data and store it in the memory. If the table data is large, the memory usage of flink tm will be relatively high, and it may even be oom. I think we should support caching some data just like jdbc connector. Are there any plans to enhance the lookup function?

banmoy commented 11 months ago

Would you like to contribute the feature?

Jin-H commented 11 months ago

I'm willing to implement this feature, but it will take some time.

banmoy commented 11 months ago

I think it's ok, and first assign this issue to you. Thanks.

Jin-H commented 11 months ago

ok