search

prestodb / rfcs

Presto RFCs
Apache License 2.0
0 stars 22 forks source link

Add RFC for supporting distributed procedure #12

Open hantangwangd opened 5 months ago

hantangwangd commented 5 months ago

Propose design to expand the current procedure architecture in presto, support defining, registering and calling procedures which need to be executed in a distributed way.

Besides, in order to demonstrate as an example and to figure out the functional boundaries among the different architecture levels, also describe the design for Iceberg to support distributed procedure as well as the design of a specific distributed procedure rewrite_data_files.

hantangwangd commented 5 months ago

@rschlussel Thanks for the comment. Yes, of course we should integrate this for cpp workers if it's proved to be useful and needed.

I think we can take a two-step approach for this. Firstly, we support it on java workers, confirm the feasible of the entire architecture and figure out the best functional boundary division. Then, after that, we can support it on cpp workers following the design and implementation path on java workers. What's your opinion?

hantangwangd commented 3 months ago

This RFC is ready for review, the comments are all handled, and the relevant implementation can be viewed here: https://github.com/prestodb/presto/pull/22659. Please take a look when convenient, thanks! cc: @rschlussel @aditi-pandit @jaystarshot @tdcmeehan @ZacBlanco @yingsu00