treasure-data / digdag

Workload Automation System
https://www.digdag.io/
Apache License 2.0
1.3k stars 221 forks source link

shell_sample.sh is evaluated as JavaScript? #3

Closed xerial closed 8 years ago

xerial commented 8 years ago

Observed the following error:

leo@weaver:~/work/td/2016-03-09> digdag new query-analysis                                                                                                     [9:13:26 Mar 07 2016]
2016-03-09 09:13:36 +0900: Digdag v0.3.4
  Creating query-analysis/digdag
  Creating query-analysis/.digdag-wrapper/digdag.jar
  Creating query-analysis/.gitignore
  Creating query-analysis/tasks/shell_sample.sh
  Creating query-analysis/tasks/repeat_hello.sh
  Creating query-analysis/tasks/__init__.py
  Creating query-analysis/digdag.yml
Done. Type `cd query-analysis` and `./digdag r` to run the workflow. Enjoy!
leo@weaver:~/work/td/2016-03-09> cd query-analysis                                                                                                             [9:13:36 Mar 07 2016]
leo@weaver:~/work/td/2016-03-09/query-analysis> ls                                                                                                             [9:13:38 Mar 07 2016]
digdag      digdag.yml  tasks
leo@weaver:~/work/td/2016-03-09/query-analysis> digdag run                                                                                                     [9:13:38 Mar 07 2016]
2016-03-09 09:13:49 +0900: Digdag v0.3.4
2016-03-09 09:13:50 +0900 [WARN] (main): --session-time argument, --hour argument, or _schedule in yaml file is not set. Using today's 00:00:00 as ${session_time}.
2016-03-09 09:13:50 +0900 [INFO] (main): Using state files at digdag.status/20160309T000000+0900.
2016-03-09 09:13:50 +0900 [INFO] (main): Starting a new session repository id=1 workflow name=+main session_time=2016-03-09T00:00:00+09:00
2016-03-09 09:13:50 +0900 [ERROR] (0021@+main+step1): Task failed
java.lang.RuntimeException: Failed to process task config templates
    at io.digdag.core.agent.OperatorManager.runWithArchive(OperatorManager.java:154)
    at io.digdag.core.agent.OperatorManager.lambda$runWithHeartbeat$1(OperatorManager.java:128)
    at io.digdag.core.agent.OperatorManager$$Lambda$114/2102246737.run(Unknown Source)
    at io.digdag.core.agent.CurrentDirectoryArchiveManager.withExtractedArchive(CurrentDirectoryArchiveManager.java:20)
    at io.digdag.core.agent.OperatorManager.runWithHeartbeat(OperatorManager.java:127)
    at io.digdag.core.agent.OperatorManager.run(OperatorManager.java:106)
    at io.digdag.cli.Run$OperatorManagerWithSkip.run(Run.java:567)
    at io.digdag.core.agent.LocalAgent.lambda$run$0(LocalAgent.java:61)
    at io.digdag.core.agent.LocalAgent$$Lambda$112/1884150000.run(Unknown Source)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: io.digdag.spi.TemplateException: Failed to evaluate JavaScript code: tasks/shell_sample.sh
    at io.digdag.core.agent.ConfigEvalEngine.invokeTemplate(ConfigEvalEngine.java:91)
    at io.digdag.core.agent.ConfigEvalEngine.access$200(ConfigEvalEngine.java:31)
    at io.digdag.core.agent.ConfigEvalEngine$Context.evalValue(ConfigEvalEngine.java:170)
    at io.digdag.core.agent.ConfigEvalEngine$Context.evalObjectRecursive(ConfigEvalEngine.java:128)
    at io.digdag.core.agent.ConfigEvalEngine$Context.access$000(ConfigEvalEngine.java:95)
    at io.digdag.core.agent.ConfigEvalEngine.eval(ConfigEvalEngine.java:62)
    at io.digdag.core.agent.OperatorManager.runWithArchive(OperatorManager.java:151)
    ... 13 common frames omitted
Caused by: javax.script.ScriptException: String index out of range: 72
    at jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:455)
    at jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:387)
    at jdk.nashorn.api.scripting.NashornScriptEngine.invokeFunction(NashornScriptEngine.java:187)
    at io.digdag.core.agent.ConfigEvalEngine.invokeTemplate(ConfigEvalEngine.java:88)
    ... 19 common frames omitted
Caused by: jdk.nashorn.internal.runtime.ParserException: String index out of range: 72
    at jdk.nashorn.internal.runtime.Context$ThrowErrorManager.error(Context.java:419)
    at jdk.nashorn.internal.parser.Parser.recover(Parser.java:413)
    at jdk.nashorn.internal.parser.Parser.sourceElements(Parser.java:831)
    at jdk.nashorn.internal.parser.Parser.program(Parser.java:711)
    at jdk.nashorn.internal.parser.Parser.parse(Parser.java:284)
    at jdk.nashorn.internal.runtime.RecompilableScriptFunctionData.reparse(RecompilableScriptFunctionData.java:386)
    at jdk.nashorn.internal.runtime.RecompilableScriptFunctionData.compileTypeSpecialization(RecompilableScriptFunctionData.java:511)
    at jdk.nashorn.internal.runtime.RecompilableScriptFunctionData.getBest(RecompilableScriptFunctionData.java:730)
    at jdk.nashorn.internal.runtime.ScriptFunctionData.getBestInvoker(ScriptFunctionData.java:232)
    at jdk.nashorn.internal.runtime.ScriptFunction.findCallMethod(ScriptFunction.java:586)
    at jdk.nashorn.internal.runtime.ScriptObject.lookup(ScriptObject.java:1872)
    at jdk.nashorn.internal.runtime.linker.NashornLinker.getGuardedInvocation(NashornLinker.java:100)
    at jdk.nashorn.internal.runtime.linker.NashornLinker.getGuardedInvocation(NashornLinker.java:94)
    at jdk.internal.dynalink.support.CompositeTypeBasedGuardingDynamicLinker.getGuardedInvocation(CompositeTypeBasedGuardingDynamicLinker.java:176)
    at jdk.internal.dynalink.support.CompositeGuardingDynamicLinker.getGuardedInvocation(CompositeGuardingDynamicLinker.java:124)
    at jdk.internal.dynalink.support.LinkerServicesImpl.getGuardedInvocation(LinkerServicesImpl.java:149)
    at jdk.internal.dynalink.DynamicLinker.relink(DynamicLinker.java:233)
    at jdk.nashorn.internal.objects.NativeRegExp.callReplaceValue(NativeRegExp.java:819)
    at jdk.nashorn.internal.objects.NativeRegExp.replace(NativeRegExp.java:696)
    at jdk.nashorn.internal.objects.NativeString.replace(NativeString.java:809)
    at jdk.nashorn.internal.scripts.Script$Recompilation$1$62AA$\^eval\_.template(<eval>:26)
    at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:640)
    at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:229)
    at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:387)
    at jdk.nashorn.api.scripting.ScriptObjectMirror.callMember(ScriptObjectMirror.java:192)
    at jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:381)
    ... 21 common frames omitted
error:
  * +main+step1:
    Failed to process task config templates

Task state is saved at digdag.status/20160309T000000+0900 directory.
Run command with --session-time '2016-03-09 00:00:00' argument to retry failed tasks.```
xerial commented 8 years ago

Upgrading jdk is necessary to fix this.

xerial commented 8 years ago

Adding java.version system property check is necessary.

frsyuki commented 8 years ago

Added version check.