[upsert] integrate partial upsert with groovy evaluator

[deemoliu]

Partial upsert let users choose to update only specific columns and ignore the rest, and apply complex merging logic based on the existing row and the new row that shares the same primary key.

Pinot allows you to run any function using Apache Groovy scripts. To support complex use cases (e.g merge previous and new values based on conditions), we proposed to enhance partial upsert with groovy transformation function.

a few challenges to consider

• 1. limitation of Groovy function regex, currently the groovy evaluator doesn't support multi-line groovy script.
• 2. FunctionEvaluator interface currently only take one GenericRow as input.
• 2.1 For partial upsert we need to access both old row and the new row.
• 2.2 How to represent the argument and how to bind the arguments with values for both previous row and the new row.
• 3. Integrate Groovy function in the partial upsert handler

[deemoliu]

hello @Jackie-Jiang, this is one of the feature we have discussed with you. can you assign this task to rohityadav1993 and me?

[Jackie-Jiang]

@deemoliu In order to assign an issue to a non-committer, he/she needs to reply to the issue

[rohityadav1993]

@Jackie-Jiang, you can assign it to me now. I will share a doc on the approach in the next few days.

[rohityadav1993]

@deemoliu @Jackie-Jiang , prepared a small doc with initial thought on how to support this: https://docs.google.com/document/d/1bBTCYZFP2stvzc6xZUOEh-XweVgC9WfD7uiSPbKtaZY/edit?usp=sharing