search

onyx-platform / onyx

Distributed, masterless, high performance, fault tolerant data processing
http://www.onyxplatform.org
Eclipse Public License 1.0
2.05k stars 204 forks source link

Lifecycle fn issues are difficult to diagnose #528

Open lbradstreet opened 8 years ago

lbradstreet commented 8 years ago

I had an issue where a lifecycle function returned a bool rather than a map, and it was rather opaque to debug where the issue was coming from. See the stack trace below:

I think we should catch and rethrow the exception in the lifecycle handling code, and dump the lifecycle and task name that caused the issue. It would also be good if we could name the anonymous function doing the merging based on the lifecycle name, however this would require a macro to wrap fn, to build anonymous named functions.

16-Feb-13 15:49:19 lbpro ERROR [onyx.peer.virtual-peer] - java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Boolean Error in processing loop. Restarting.
                                                 java.lang.Thread.run              Thread.java:  745
                   java.util.concurrent.ThreadPoolExecutor$Worker.run  ThreadPoolExecutor.java:  617
                    java.util.concurrent.ThreadPoolExecutor.runWorker  ThreadPoolExecutor.java: 1142
                                                                  ...
                                    clojure.core.async/thread-call/fn                async.clj:  434
                                onyx.peer.virtual-peer.VirtualPeer/fn         virtual_peer.clj:  106
                               onyx.peer.virtual-peer/processing-loop         virtual_peer.clj:   24
                  onyx.peer.virtual-peer/processing-loop/invokeStatic         virtual_peer.clj:   57
                                                                  ...
                            onyx.log.commands.seal-output/eval20897/G          seal_output.clj:   46
                         onyx.log.commands.seal-output/eval20897/G/fn          seal_output.clj:   48
                         onyx.log.commands.common/start-new-lifecycle               common.clj:  132
            onyx.log.commands.common/start-new-lifecycle/invokeStatic               common.clj:  138
                                            onyx.system.OnyxTask/stop               system.clj:  146
                                        onyx.system/rethrow-component               system.clj:   58
                           onyx.system/rethrow-component/invokeStatic               system.clj:   60
                                              onyx.system.OnyxTask/fn               system.clj:  147
                        com.stuartsierra.component$stop_system.invoke           component.cljc:  165
                  com.stuartsierra.component$stop_system.invokeStatic           component.cljc:  173
                                                                  ...
            com.stuartsierra.component$update_system_reverse.doInvoke           component.cljc:  143
        com.stuartsierra.component$update_system_reverse.invokeStatic           component.cljc:  147
                                                  clojure.core/reduce                 core.clj: 6527
                                     clojure.core/reduce/invokeStatic                 core.clj: 6544
                                                                  ...
    com.stuartsierra.component$update_system_reverse$fn__13111.invoke           component.cljc:  151
                         com.stuartsierra.component$try_action.invoke           component.cljc:  116
                   com.stuartsierra.component$try_action.invokeStatic           component.cljc:  117
                                                   clojure.core/apply                 core.clj:  641
                                      clojure.core/apply/invokeStatic                 core.clj:  648
                                                                  ...
com.stuartsierra.component$eval13049$fn__13061$G__13040__13066.invoke           component.cljc:    5
com.stuartsierra.component$eval13049$fn__13061$G__13041__13063.invoke           component.cljc:    5
                          onyx.peer.task-lifecycle.TaskLifeCycle/stop       task_lifecycle.clj:  616
                                                 clojure.core/comp/fn                 core.clj: 2460
  onyx.lifecycles.lifecycle-compile/compile-lifecycle-functions/fn/fn    lifecycle_compile.clj:   65
                                                                  ...
                                                   clojure.core/merge                 core.clj: 2932
                                      clojure.core/merge/invokeStatic                 core.clj: 2939
                                    clojure.core/reduce1/invokeStatic                 core.clj:  915
                                    clojure.core/reduce1/invokeStatic                 core.clj:  925
                                                clojure.core/merge/fn                 core.clj: 2940
                                       clojure.core/conj/invokeStatic                 core.clj:   85
                                                                  ...
java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Boolean
lbradstreet commented 8 years ago

Some schema checks sprinkled around here would also help when running tests.