Many of our Spark extending libraries need a way to report errors that emerged during data processing.
They can be either re-implementing the preferred method of the application their main usage is expected in, or having a possible different way of error reporting than the application or simply ignore the error.
This Spike is to investigate, if it would be possible to create a generic trait to report the errors. This traits different implementation then would be provided to the library upon initialization by the higher level app therefore decoupling the style of error handling from the library that needs it.
Questions To Answer
Can such a generic trait be effectively created
Would that give enough flexibility and anchor-points at the same time
Desired Outcome
Have a main trait and system of supporting classes for general error handling.
### Tasks
- [x] Implement the main `trait ErrorHandling` and necessary support classes to provide general error handling
- [x] Refactor `ErrorMessage` class
- [x] Implement error handling by putting the info into column of `ErrorMessage` array
- [ ] https://github.com/AbsaOSS/spark-commons/issues/88
- [ ] https://github.com/AbsaOSS/spark-commons/issues/89
- [ ] https://github.com/AbsaOSS/spark-commons/issues/91
- [ ] https://github.com/AbsaOSS/spark-commons/issues/101
- [ ] https://github.com/AbsaOSS/spark-commons/issues/93
- [ ] #111
Background
Many of our Spark extending libraries need a way to report errors that emerged during data processing. They can be either re-implementing the preferred method of the application their main usage is expected in, or having a possible different way of error reporting than the application or simply ignore the error.
This Spike is to investigate, if it would be possible to create a generic
trait
to report the errors. Thistrait
s different implementation then would be provided to the library upon initialization by the higher level app therefore decoupling the style of error handling from the library that needs it.Questions To Answer
trait
be effectively createdDesired Outcome
Have a main
trait
and system of supporting classes for general error handling.Depends on #82