Closed mindflow94 closed 2 years ago
I tried to modify shellEngineConnExecutor's code and this issue was resolved.
val processBuilder: ProcessBuilder = new ProcessBuilder(generateRunCodeToList(code))
private def generateRunCodeToList(code: String): util.List[String] = {
util.Arrays.asList("sh", "-c", code)
}
I didn't do the full test and didn't know if there would be any other risks.
I tried to modify shellEngineConnExecutor's code and this issue was resolved.
val processBuilder: ProcessBuilder = new ProcessBuilder(generateRunCodeToList(code)) private def generateRunCodeToList(code: String): util.List[String] = { util.Arrays.asList("sh", "-c", code) }
I didn't do the full test and didn't know if there would be any other risks.
This code seems to just change the scala.Array
type to the java.util.List
type. Can you display specific error messages?
Can you display specific error messages
Toggle the constructor of the ProcessBuilder
, Pass List<String>
instead of String...
, To avoid parsing special symbols in scripts.
There is no explicit error message, just a prompt for the shell code is wrong.The following is a detailed error:
2021-12-02 14:24:04.538 ERROR [Linkis-Default-Scheduler-Thread-5] com.webank.wedatasphere.linkis.manager.engineplugin.shell.executor.ShellEngineConnExecutor 61 error - exitCode is 1
2021-12-02 14:24:04.539 ERROR [Linkis-Default-Scheduler-Thread-5] com.webank.wedatasphere.linkis.manager.engineplugin.shell.executor.ShellEngineConnExecutor 61 error -
2021-12-02 14:24:04.555 ERROR [Linkis-Default-Scheduler-Thread-5] com.webank.wedatasphere.linkis.manager.engineplugin.shell.executor.ShellEngineConnExecutor 57 error - execute code failed! com.webank.wedatasphere.linkis.manager.engineplugin.shell.exception.ShellCodeErrorException: errCode: 50037 ,desc: shell code is wrong ,ip: ct6 ,port: 44542 ,serviceKind: linkis-cg-engineconn
at com.webank.wedatasphere.linkis.manager.engineplugin.shell.executor.ShellEngineConnExecutor.executeLine(ShellEngineConnExecutor.scala:88) ~[linkis-engineplugin-shell-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$toExecuteTask$2$$anonfun$apply$10$$anonfun$apply$11.apply(ComputationExecutor.scala:179) ~[linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$toExecuteTask$2$$anonfun$apply$10$$anonfun$apply$11.apply(ComputationExecutor.scala:178) ~[linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:39) ~[linkis-common-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$toExecuteTask$2$$anonfun$apply$10.apply(ComputationExecutor.scala:180) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$toExecuteTask$2$$anonfun$apply$10.apply(ComputationExecutor.scala:174) [linkis-computation-engineconn-1.0.2.jar:?]
at scala.collection.immutable.Range.foreach(Range.scala:160) [scala-library-2.11.12.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$toExecuteTask$2.apply(ComputationExecutor.scala:173) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$toExecuteTask$2.apply(ComputationExecutor.scala:149) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:60) [linkis-common-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.toExecuteTask(ComputationExecutor.scala:222) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$3.apply(ComputationExecutor.scala:237) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor$$anonfun$3.apply(ComputationExecutor.scala:237) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:60) [linkis-common-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.acessible.executor.entity.AccessibleExecutor.ensureIdle(AccessibleExecutor.scala:54) [linkis-accessible-executor-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.acessible.executor.entity.AccessibleExecutor.ensureIdle(AccessibleExecutor.scala:48) [linkis-accessible-executor-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.ensureOp(ComputationExecutor.scala:133) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.execute(ComputationExecutor.scala:236) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.com$webank$wedatasphere$linkis$engineconn$computation$executor$service$TaskExecutionServiceImpl$$executeTask(TaskExecutionServiceImpl.scala:239) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl$$anon$1$$anonfun$run$1.apply$mcV$sp(TaskExecutionServiceImpl.scala:172) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl$$anon$1$$anonfun$run$1.apply(TaskExecutionServiceImpl.scala:170) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl$$anon$1$$anonfun$run$1.apply(TaskExecutionServiceImpl.scala:170) [linkis-computation-engineconn-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:39) [linkis-common-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryAndWarn(Utils.scala:68) [linkis-common-1.0.2.jar:?]
at com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl$$anon$1.run(TaskExecutionServiceImpl.scala:170) [linkis-computation-engineconn-1.0.2.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_281]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_281]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_281]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_281]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_281]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_281]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]
The two construction methods seem to be equivalent. I tested the shell and found no errors. Is it caused by others?
ls /home/hadoop > /home/hadoop/tmp.txt
For me, with the changes above, my Shell Engine can successfully execute ls > tmp.txt
. If you can pass the ls /home/hadoop > /home/hadoop/tmp.txt
test, I can't think of any other reason.
My linkis version is 1.0.2. When I use the shell engine to submit a shell job, the contents of the job are
ls > tmp.txt
. This job will report a mistake. I think some special operation can cause a job to fail, such as|
、>
.