search

rescalante-lilly / ruffus

Automatically exported from code.google.com/p/ruffus
MIT License
0 stars 0 forks source link

Early exception / error reporting #43

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
The pipeline does not halt, and exceptions are not forwarded until the pipeline 
has collected NNN jobs worth of exceptions where NNN = multiprocess.

Even if this behaviour is retained, the text of the errors should be printed 
out immediately.

Submitted by A. Heger

Original issue reported on code.google.com by bunbu...@gmail.com on 11 Mar 2011 at 4:29

GoogleCodeExporter commented 9 years ago 
Fixed in v. 2.3

pipeline_run(..., exceptions_terminate_immediately = False, log_exceptions = 
False)

===================================================
1) Halting pipeline early on the first exception
===================================================
By default ruffus accumulates NN errors before interrupting the pipeline 
prematurely where NN is the specified parallelism for pipeline_run. 

The pipeline will only be interrupted immediately if exceptions of
type 'ruffus.JobSignalledBreak' are thrown.

If exceptions_terminate_immediately = True, all exceptions interrupt 
immediately.

===================================================
2) Printing exception messages early
===================================================
By default, Ruffus collects all raised exceptions, and then throws them
in ensemble at the end of the pipeline. If this exception is not caught,
then the characteristic Ruffus stack traces (one per exception) will be
printed at this point.

If "log_exceptions = True" then, as each exception appears, logger.error(...) 
will
be invoked with the string representation of the each exception, and the 
associated stack trace.

The default logger prints to sys.stderr, but this can be changed to anything 
from the logging module or a compatible object via pipeline_run(..., logger = 
???)

The default logger has added the "ERROR:" and the indentation to the following 
example (edited for brevity):

ERROR:

   Exception #1
      'my_exception_type(custom_parameters)' raised in ...
       Task = def a_task_func(...):
       Job  = [i_param, o_param_something, ...]

    Traceback (most recent call last):
      File "...
    my_exception_type : custom_parameters

ERROR:

    Exception #2
      'my_exception_type(more_stuff)' raised in ...
       Task = def other_task_func(...):
       Job  = [i_param, o_param_something, ...]

    Traceback (most recent call last):
      File "...
    my_exception_type : more_stuff

Original comment by bunbu...@gmail.com on 4 Oct 2011 at 11:29

GoogleCodeExporter commented 9 years ago

Original comment by bunbu...@gmail.com on 23 Nov 2013 at 1:53