I have only one function, which takes json as parameter and do different things based on the json parameter. So I have only one jar which is quite big.
The current implement will copy the jar from mist master to all executors in the cluster for each run of the function, and take a lot of disk space, unnecessarily in this case.
It will be good to allow user to specify the jar path in cluster and omit the jar copying step, or even better, create a method to publish the artifact jar to the cluster, and then omit jar copying steps for this particular function.
I have only one function, which takes json as parameter and do different things based on the json parameter. So I have only one jar which is quite big.
The current implement will copy the jar from mist master to all executors in the cluster for each run of the function, and take a lot of disk space, unnecessarily in this case.
It will be good to allow user to specify the jar path in cluster and omit the jar copying step, or even better, create a method to publish the artifact jar to the cluster, and then omit jar copying steps for this particular function.