search

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.39k stars 4.49k forks source link

[Bug] [dolphinscheduler-task-java] 3.2.1-release version Java Task not working, need remove the ExecutePath and only use the absolute path in shell script #16211

Closed lanxing2 closed 2 days ago

lanxing2 commented 4 days ago

Search before asking

What happened

3.2.1-release version java tasks always failed. javac and java command cannot find the correct jar file and .java file After check the shell script file, I found out that the path to the jar file and .java file is not correct, the execution path appears twice and the javac and java cannot find the file correctly Looks like the code to generate the command is adding the ExecutePath and AbsolutePathInLocal, and the path is not correct I make some change in my local branch(remove the ExecutionPath when generate the jar command, the log related code is for my local debugging) and fix both Jar Model Java Task and Java Model Java Task 1719306598746 1719306628298

What you expected to happen

Create the Shell script to run the Java Task correctly

How to reproduce

For 3.2.1-RELEASE Version Create any simple Process with a Simple JavaTask and run the Process

Anything else

Looks like the dev branch is fixing this problem, but I think the test code need to be improved to prevent such problem in a RELEASE Version

Version

3.2.x

Are you willing to submit PR?

Code of Conduct

ruanwenjun commented 2 days ago

duplicated with https://github.com/apache/dolphinscheduler/issues/15902

lanxing2 commented 1 day ago

I deployed the code and did some test for Java Task in Jar Model.
Current Code will fail and if I remove ".append(taskRequest.getExecutePath()).append(FOLDER_SEPERATOR)", the task will succeed.
The following content are logs on my cluster

Current code

[INFO] 2024-06-28 16:13:02.398 +0800 - #!/bin/bash BASEDIR=$(cd dirname $0; pwd) cd $BASEDIR ${JAVA_HOME}/bin/java -classpath .:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155/libs/jnpf-dolphin-java-task-print-args.jar -jar /tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155//tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155/libs/jnpf-dolphin-java-task-print-args.jar 123 456 /tmp/user [INFO] 2024-06-28 16:13:02.398 +0800 - ** Script Content ***** [INFO] 2024-06-28 16:13:02.399 +0800 - Executing shell command : sudo -u dolphinscheduler -i /tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155/146_155.sh [INFO] 2024-06-28 16:13:02.402 +0800 - process start, process id is: 3605242 [INFO] 2024-06-28 16:13:03.402 +0800 - -> Error: Unable to access jarfile /tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155//tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155/libs/jnpf-dolphin-java-task-print-args.jar [INFO] 2024-06-28 16:13:03.403 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/146/155, processId:3605242 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1

After Remove .append(taskRequest.getExecutePath()).append(FOLDER_SEPERATOR)

[INFO] 2024-06-28 16:24:01.829 +0800 - #!/bin/bash BASEDIR=$(cd dirname $0; pwd) cd $BASEDIR ${JAVA_HOME}/bin/java -classpath .:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/148/157:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/148/157/libs/jnpf-dolphin-java-task-print-args.jar -jar /tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/148/157/libs/jnpf-dolphin-java-task-print-args.jar 123 456 /tmp/user [INFO] 2024-06-28 16:24:01.829 +0800 - ** Script Content ***** [INFO] 2024-06-28 16:24:01.830 +0800 - Executing shell command : sudo -u dolphinscheduler -i /tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/148/157/148_157.sh [INFO] 2024-06-28 16:24:01.838 +0800 - process start, process id is: 3607346 [INFO] 2024-06-28 16:24:02.839 +0800 - -> Run jnpf dolphin scheduler java sample task args 0 is 123 args 1 is 456 args 2 is /usr/root/local [INFO] 2024-06-28 16:24:02.840 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13289909089664/14095886576288_4/148/157, processId:3607346 ,exitStatusCode:0 ,processWaitForStatus:true ,processExitValue:0