search

Sage-Bionetworks / SynapseWorkflowHook

Code for linking a workflow engine to a Synapse evaluation queue
Apache License 2.0
4 stars 1 forks source link

Not binding submissionId #3

Closed thomasyu888 closed 6 years ago

thomasyu888 commented 6 years ago

| => docker-compose up WARNING: The DATA_UNLOCK_SYNAPSE_PRINCIPAL_ID variable is not set. Defaulting to a blank string. Starting synapseworkflowhook_workflow-hook_1 ... done Attaching to synapseworkflowhook_workflow-hook_1 workflow-hook_1 | [INFO] Scanning for projects... workflow-hook_1 | [INFO] workflow-hook_1 | [INFO] ------------------< org.sagebionetworks:WorkflowHook >------------------ workflow-hook_1 | [INFO] Building WorkflowHook 1.0-SNAPSHOT workflow-hook_1 | [INFO] --------------------------------[ jar ]--------------------------------- workflow-hook_1 | [INFO] workflow-hook_1 | [INFO] >>> exec-maven-plugin:1.2.1:java (default-cli) > validate @ WorkflowHook >>> workflow-hook_1 | [INFO] workflow-hook_1 | [INFO] <<< exec-maven-plugin:1.2.1:java (default-cli) < validate @ WorkflowHook <<< workflow-hook_1 | [INFO] workflow-hook_1 | [INFO] workflow-hook_1 | [INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ WorkflowHook --- workflow-hook_1 | ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. workflow-hook_1 | [org.sagebionetworks.WorkflowHook.main()] INFO org.sagebionetworks.WorkflowHook - Precheck completed successfully. workflow-hook_1 | [org.sagebionetworks.WorkflowHook.main()] INFO org.sagebionetworks.WorkflowHook - Top level loop: checking progress or starting new job. workflow-hook_1 | 22:50:27.549 [org.sagebionetworks.WorkflowHook.main()] ERROR org.sagebionetworks.client.SynapseProfileProxy - java.lang.reflect.InvocationTargetException workflow-hook_1 | Sep 19, 2018 10:50:27 PM org.sagebionetworks.ExponentialBackoffRunner execute workflow-hook_1 | SEVERE: Found status code 409. Will not retry: Status Code: 409 message: An entity with the name: 3324230 already exists with a parentId: syn16809914 workflow-hook_1 | 22:50:28.256 [org.sagebionetworks.WorkflowHook.main()] ERROR org.sagebionetworks.client.SynapseProfileProxy - java.lang.reflect.InvocationTargetException workflow-hook_1 | Sep 19, 2018 10:50:28 PM org.sagebionetworks.ExponentialBackoffRunner execute workflow-hook_1 | SEVERE: Will not retry: The requested ACL does not exist. This entity inherits its permissions from: /entity/syn16809906/acl workflow-hook_1 | [WARNING] workflow-hook_1 | java.lang.reflect.InvocationTargetException workflow-hook_1 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) workflow-hook_1 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) workflow-hook_1 | at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) workflow-hook_1 | at java.lang.reflect.Method.invoke (Method.java:566) workflow-hook_1 | at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:297) workflow-hook_1 | at java.lang.Thread.run (Thread.java:834) workflow-hook_1 | Caused by: java.lang.IllegalStateException: Running workflow(s) without corresponding submission(s): [WorkflowJobImpl [submissionId=null, containerName=workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6, container=com.github.dockerjava.api.model.Container@5054201[command=toil-cwl-runner --defaultMemory 100M --retryCount 0 --defaultDisk 1000000 --workDir /Users/ThomasY/sandbox/896e0c07-9953-4245-b00a-961e3a7f2159 --noLinkImports SynapseChallengeTemplates-develop/workflow/workflow-entrypoint.cwl /Users/ThomasY/sandbox/896e0c07-9953-4245-b00a-961e3a7f2159/TMP13218566508755787978.yml,created=1537397192,id=8166023b1acce4b189e193ca843e31e56416471b3be0fcdda19e683529a7589e,image=docker.synapse.org/syn5644795/docker-and-toil,imageId=sha256:6396af54e4ed1924b770ee96fb92f9fa75f67a5f4042ff1bcc89efe1765a811b,names={/workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6},ports={},labels={},status=Exited (0) 3 minutes ago,sizeRw=null>,sizeRootFs=<null>,hostConfig=com.github.dockerjava.api.model.ContainerHostConfig@720b17f2[networkMode=default],networkSettings=com.github.dockerjava.api.model.ContainerNetworkSettings@54b97149[networks={bridge=com.github.dockerjava.api.model.ContainerNetwork@51cc423e[ipamConfig=<null,links=,aliases=,networkID=49fa757a2da1d586162745aa324be69cb99ec54b4c00cdd49312afb414700a8f,endpointId=,gateway=,ipAddress=,ipPrefixLen=0,ipV6Gateway=,globalIPv6Address=,globalIPv6PrefixLen=0,macAddress=]}]], submissionParameters=null]] workflow-hook_1 | at org.sagebionetworks.WorkflowHook.updateWorkflowJobs (WorkflowHook.java:308) workflow-hook_1 | at org.sagebionetworks.WorkflowHook.execute (WorkflowHook.java:185) workflow-hook_1 | at org.sagebionetworks.WorkflowHook.main (WorkflowHook.java:103) workflow-hook_1 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) workflow-hook_1 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) workflow-hook_1 | at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) workflow-hook_1 | at java.lang.reflect.Method.invoke (Method.java:566) workflow-hook_1 | at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:297) workflow-hook_1 | at java.lang.Thread.run (Thread.java:834) workflow-hook_1 | [INFO] ------------------------------------------------------------------------ workflow-hook_1 | [INFO] BUILD FAILURE workflow-hook_1 | [INFO] ------------------------------------------------------------------------ workflow-hook_1 | [INFO] Total time: 13.952 s workflow-hook_1 | [INFO] Finished at: 2018-09-19T22:50:35Z workflow-hook_1 | [INFO] ------------------------------------------------------------------------ workflow-hook_1 | [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project WorkflowHook: An exception occured while executing the Java class. null: InvocationTargetException: Running workflow(s) without corresponding submission(s): [WorkflowJobImpl [submissionId=null, containerName=workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6, container=com.github.dockerjava.api.model.Container@5054201[command=toil-cwl-runner --defaultMemory 100M --retryCount 0 --defaultDisk 1000000 --workDir /Users/ThomasY/sandbox/896e0c07-9953-4245-b00a-961e3a7f2159 --noLinkImports SynapseChallengeTemplates-develop/workflow/workflow-entrypoint.cwl /Users/ThomasY/sandbox/896e0c07-9953-4245-b00a-961e3a7f2159/TMP13218566508755787978.yml,created=1537397192,id=8166023b1acce4b189e193ca843e31e56416471b3be0fcdda19e683529a7589e,image=docker.synapse.org/syn5644795/docker-and-toil,imageId=sha256:6396af54e4ed1924b770ee96fb92f9fa75f67a5f4042ff1bcc89efe1765a811b,names={/workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6},ports={},labels={},status=Exited (0) 3 minutes ago,sizeRw=null>,sizeRootFs=<null>,hostConfig=com.github.dockerjava.api.model.ContainerHostConfig@720b17f2[networkMode=default],networkSettings=com.github.dockerjava.api.model.ContainerNetworkSettings@54b97149[networks={bridge=com.github.dockerjava.api.model.ContainerNetwork@51cc423e[ipamConfig=<null,links=,aliases=,networkID=49fa757a2da1d586162745aa324be69cb99ec54b4c00cdd49312afb414700a8f,endpointId=,gateway=,ipAddress=,ipPrefixLen=0,ipV6Gateway=,globalIPv6Address=,globalIPv6PrefixLen=0,macAddress=]}]], submissionParameters=null]] -> [Help 1] workflow-hook_1 | [ERROR] workflow-hook_1 | [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. workflow-hook_1 | [ERROR] Re-run Maven using the -X switch to enable full debug logging. workflow-hook_1 | [ERROR] workflow-hook_1 | [ERROR] For more information about the errors and possible solutions, please read the following articles: workflow-hook_1 | [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException workflow-hook_1 | [Thread-1] INFO org.sagebionetworks.ShutdownHook - Shut down signal received. workflow-hook_1 | [Thread-1] INFO org.sagebionetworks.ShutdownHook - Shut down complete. synapseworkflowhook_workflow-hook_1 exited with code 1

brucehoff commented 6 years ago

The actual error in this log is:

Caused by: java.lang.IllegalStateException: Running workflow(s) without corresponding submission(s): [WorkflowJobImpl [submissionId=null, containerName=workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6

This just means you have a 'stray' job container, called workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6. One way to recover is to delete the job:

docker rm workflow_job.a8e5ce14-8b3b-460b-babe-39e849cea5a6

Should update the error message with this explanation.

brucehoff commented 6 years ago

https://github.com/Sage-Bionetworks/SynapseWorkflowHook/commit/765d4efcdc8e73beae7fa9bec2533b27349848fe