Closed lanxing2 closed 1 month ago
In JavaTask, MainArgs and JVMArgs are use directly without Placeholder Replacement https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-task-plugin/dolphinscheduler-task-java/src/main/java/org/apache/dolphinscheduler/plugin/task/java/JavaTask.java#L169-L170 https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-task-plugin/dolphinscheduler-task-java/src/main/java/org/apache/dolphinscheduler/plugin/task/java/JavaTask.java#L190-L191
In SparkTask, MainArgs and others are use directly without Placeholder Replacement https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java#L198 https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java#L180
In SparkTask, MainArgs and others are use directly without Placeholder Replacement https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java#L198 https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java#L180
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
Search before asking
Description
Currently, for task-java
ParameterUtils.convertParameterPlaceholders will replace placeholder for the Java Code if Java Task run in the Java Mode.
However if run the Java Task in Jar Mode, I cannot pass Parameters to the task, because in the mainArgs and jvmArgs do not have ParameterUtils.convertParameterPlaceholders to replace the placeholders with parameter value.
If we enable placeholder replacement for args, JavaTask in Jar Mode can get the value of Parameters in args.
Similiarily, for task-spark
ParameterUtils.convertParameterPlaceholders will replace placeholder for the SQL Code if Java Task run in the SQLMode.
However if run the Spark Task in other Mode, cannot pass the parameters to the task, because mainArgs and otherArgs do not have ParameterUtils.convertParameterPlaceholders to replace the placeholders with parameter value.
If we enable placeholder replacement for args, SparkTask in Mode other than SQL can get the value of Parameters in args.
I purpose this Improvement because in my team, we save some common data such in project parameters.
And use these parameters in different processes and tasks, then I notice JavaTask in Jar Mode and Spark Task not in SQL Mode cannot use these parameters. I actually add placeholder replacement for Java Task in Jar Mode in our teams' cluster.
Hope the core member agree with this purposal, I can submit PR for this feature/improvement.
Are you willing to submit a PR?
Code of Conduct