search

gschueler / rundeck-log4j-streaming-log-plugin

0 stars 0 forks source link

Amiguous overloading stalls jobs #3

Closed ameissnersofi closed 7 years ago

ameissnersofi commented 7 years ago

Lately, I've been having issues where my scheduled rundeck jobs never terminate. After killing a stalled job, I'm presenting with these logs:

Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]: 2016-12-10 17:21:56,453 - [ameissner] [Monitors] [/Fleet_Monitor] [#605071] [fe49691a-fead-4e94-9449-befa22b03714] - Result: 2147483647
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]: 2016-12-10 17:21:56,453 project: Monitors execution: 605071 (log, ) /Fleet_Monitor [fe49691a-fead-4e94-9449-befa22b03714] - Result: 2147483647
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]: 2016-12-10 17:21:56,859 - [ameissner] [Monitors] [/Fleet_Monitor] [#605071] [fe49691a-fead-4e94-9449-befa22b03714] - Failed: NonZeroResultCode: Result code was 2147483647
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]: 2016-12-10 17:21:56,863 - [] [] [/] [#] [] - Execution failed: 605071 in project Monitors: [Workflow result: , step failures: {1=Dispatch failed on 1 nodes: [internal.sofi.com: NonZeroResultCode: Result code was 2147483647]}, Node failures: {internal.sofi.com=[NonZeroResultCode: Result code was 2147483647]}, status: failed]
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]: 2016-12-10 17:21:56,863 - [] [] [/] [#] [] - Failed addEvent for StreamingLogWriter(log4jStreamingLogWriterPlugin), disabling. Ambiguous method overloading for method java.util.HashMap#plus.
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]: Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]:         [interface java.util.Collection]
Dec 10 17:21:56 infra-2 build.sofi.com:5000/rundeck[28030]:         [interface java.util.Map]
Dec 10 17:21:57 infra-2 build.sofi.com:5000/rundeck[28030]: 2016-12-10 17:21:56,859 project: Monitors execution: 605071 (log, ) /Fleet_Monitor [fe49691a-fead-4e94-9449-befa22b03714] - Failed: NonZeroResultCode: Result code was 2147483647

It looks like the ambiguous reference is maybe preventing rundeck from putting the job in a finished state, successful or otherwise. I've currently got jobs that have been running all night, even though the processes are complete.

ameissnersofi commented 7 years ago

This was a red herring. Sorry for the noise. After removing this plugin the problem persisted.