search

rundeck-plugins / rundeck-winrm-plugin

Rundeck WinRM Node Executor plugin
Other
56 stars 28 forks source link

Randomly getting NoClassDefFoundError #43

Closed etlweather closed 5 years ago

etlweather commented 8 years ago

Rundeck 2.6.8-1 winRM Plugin: 1.3.3

Stacktrace:

[workflow] finishExecuteNodeStep(serverA): NodeDispatch: Unknown: com/xebialabs/overthere/local/LocalConnection
20:11:28            Failed dispatching to node serverA: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection
20:11:28            Failed dispatching to node serverA: com.dtolabs.rundeck.core.execution.workflow.steps.node.NodeStepException: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeNodeStep(ExecutionServiceImpl.java:158)
    at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:116)
    at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:58)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.dispatchToNodes(ExecutionServiceImpl.java:177)
    at com.dtolabs.rundeck.core.execution.workflow.steps.NodeDispatchStepExecutor.executeWorkflowStep(NodeDispatchStepExecutor.java:66)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeStep(ExecutionServiceImpl.java:116)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowStrategy.executeWFItem(BaseWorkflowStrategy.java:225)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowStrategy.executeWorkflowItemsForNodeSet(BaseWorkflowStrategy.java:303)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowStrategy.executeWorkflowItemsForNodeSet(BaseWorkflowStrategy.java:258)
    at com.dtolabs.rundeck.core.execution.workflow.StepFirstWorkflowStrategy.executeWorkflowImpl(StepFirstWorkflowStrategy.java:79)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowStrategy.executeWorkflow(BaseWorkflowStrategy.java:182)
    at com.dtolabs.rundeck.core.execution.workflow.NodeFirstWorkflowStrategy$DispatchedWorkflow.dispatch(NodeFirstWorkflowStrategy.java:383)
    at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:118)
    at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:64)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.dispatchToNodes(ExecutionServiceImpl.java:196)
    at com.dtolabs.rundeck.core.execution.workflow.NodeFirstWorkflowStrategy.executeWFSectionNodeDispatch(NodeFirstWorkflowStrategy.java:189)
    at com.dtolabs.rundeck.core.execution.workflow.NodeFirstWorkflowStrategy.executeWorkflowImpl(NodeFirstWorkflowStrategy.java:110)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowStrategy.executeWorkflow(BaseWorkflowStrategy.java:182)
    at com.dtolabs.rundeck.core.execution.WorkflowExecutionServiceThread.run(WorkflowExecutionServiceThread.java:55)
Caused by: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection
    at com.xebialabs.overthere.spi.BaseOverthereConnection.close(BaseOverthereConnection.java:132)
    at com.dtolabs.rundeck.plugin.overthere.OTWinRMNodeExecutor.executeCommand(OTWinRMNodeExecutor.java:244)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeCommand(ExecutionServiceImpl.java:333)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeCommand(ExecutionServiceImpl.java:307)
    at com.dtolabs.rundeck.core.execution.workflow.steps.node.impl.ExecNodeStepExecutor.executeNodeStep(ExecNodeStepExecutor.java:53)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeNodeStep(ExecutionServiceImpl.java:149)
    ... 18 more
Caused by: java.lang.ClassNotFoundException: com.xebialabs.overthere.local.LocalConnection
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:424)
    at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:377)
    at com.dtolabs.rundeck.core.plugins.LocalFirstClassLoader.loadClass(LocalFirstClassLoader.java:42)
    at com.dtolabs.rundeck.core.plugins.LocalFirstClassLoader.loadClass(LocalFirstClassLoader.java:28)
    ... 24 more

The exact same job, no changes, will work a few moment later.

etlweather commented 8 years ago

This job, did all its steps but then failed at the end with this.

00:48:48    rundeck-server.local    12. Command [workflow] finishExecuteNodeStep(ts32a): NodeDispatch: Unknown: com/xebialabs/overthere/local/LocalConnection
00:48:48            12: Workflow step finished, result: Dispatch failed on 2 nodes: [ts32a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection, ts30a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]
00:48:48            [workflow] Finish step: 12,NodeDispatch
00:48:48            [workflow] Finish execution:  rundeck-workflow-step-first: [Workflow result: , step failures: {12=Dispatch failed on 2 nodes: [ts32a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection, ts30a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]}, Node failures: {ts32a=[Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection], ts30a=[Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]}, flow control: Continue, status: failed]
00:48:48            [Workflow result: , step failures: {12=Dispatch failed on 2 nodes: [ts32a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection, ts30a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]}, Node failures: {ts32a=[Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection], ts30a=[Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]}, flow control: Continue, status: failed]
00:48:48            Execution failed: 4568: [Workflow result: , step failures: {12=Dispatch failed on 2 nodes: [ts32a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection, ts30a: Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]}, Node failures: {ts32a=[Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection], ts30a=[Unknown: java.lang.NoClassDefFoundError: com/xebialabs/overthere/local/LocalConnection]}, flow control: Continue, status: failed]
gschueler commented 8 years ago

this may be related to a fix coming in rundeck 2.6.9 https://github.com/rundeck/rundeck/issues/1898

etlweather commented 8 years ago

Might be - would make sense as it look like that. I run on Linux by the way, with deb install. Is there an immediate workaround? Or when should we expect 2.6.9?

gschueler commented 8 years ago

eta soon (~this week)

etlweather commented 8 years ago

eta: ok, that's cool!

etlweather commented 8 years ago

This hasn't occurred since I upgraded to 2.6.9.

etlweather commented 8 years ago

Unfortunately, it occurred again on a job last night. If that makes any difference, that job takes about 3h 20m to run.

etlweather commented 8 years ago

Should I move this to Rundeck? Seems more like a Rundeck bug than the WinRM Plugin.