NDCLab / pepper-pipeline

tool | Python Easy Pre-Processing EEG Reproducible Pipeline
GNU Affero General Public License v3.0
3 stars 3 forks source link

Dev improve exceptions #395

Closed F-said closed 2 years ago

F-said commented 2 years ago
yanbin-niu commented 2 years ago

@F-said what is the difference for this part? I think the old way also try exception all errors. If the old way does not work, how this new way can work. Sorry if I miss something.

image
DMRoberts commented 2 years ago

@yanbin-niu I believe the difference is that in the old way, we are checking for exceptions or error conditions at specific times within each function, catching them, and returning the appropriate error message in the output dictionary labeled as an error. However because we are catching the exceptions within the functions, they aren't propagating up to run.py, the functions return normally. But on the other hand if exceptions that we didn't anticipate are thrown outside of a try/catch inside the function, those are being propagated up to run.py and written to a log file instead of the output dictionary as error.

F-said commented 2 years ago

@yanbin-niu The part that I wrote originally was redundant. Notice how they essentially did the same thing:

if there was an exception
    clean outputs
    break to next subject

I figured that since this approach is limited in terms of writing all errors to the dictionary & the structure was sort of convoluted, it would be better to just allow all unhandled exceptions to get thrown and handle them in run.py when it comes to writing out what went wrong.

As we iterate over preprocess we may want to bring back the exception handling in individual features.

yanbin-niu commented 2 years ago

@DMRoberts @F-said Thanks for explaining. I really appreciate that!

I am still trying to understand the code.

Here is what I understand how our old way works:

image
  1. if an exception is captured by a feature, we will go the "2" part, then we print CAUGHT_EXCEPTION_SKIP on screen and do the clean_outputs, then go the next preprocess step;
  2. if there is unexpected exception, we will go the "1" part, then we print on screen e and traceback.format_exc(), then go the next preprocess step; right?? And both break only break the step loop, not the participant loop, right?

So, technically speaking, the old way can catch the unexpected exception. It just cannot write into our output dictionary. right?

yanbin-niu commented 2 years ago

@DMRoberts t

those are being propagated up to run.py and written to a log file instead of the output dictionary as error.

we only print to screen and not actually write them into a log file, right? Or I miss something....