vitrivr's next-generation retrieval engine. It is capable of extracting and retrieving a wider range of multimedia objects such as audio, video, images or 3d models.
Error handling in vitrivr-engine currently has two major shortcomings.
An Operator's implementer decides, if an error should be handled gracefully (i.e., log and continue) or not (i.e., throw an exception). This leads to inconsistent behaviour across the a pipeline.
If an error is logged, the caller of a pipeline has no way to access error information since most of the time, errors are simply logged. This is not ideal in cases, where vitrivr-engine is used as a library rather than a local service.
I therefore propose three major changes to how errors should be handled:
In case an error occurs, operators throw an ExtractionException. This exception reports on the error condition (retrievable, name of the operator and cause) and (optionally) wraps downstream exceptions. Throwing any other exception from within an Operator is considered a programmer's error. Therefore, proper exception handling is needed.
When configuring a pipeline, one can determine what error handling mode should be employed. Currently I see two modes: CONTINUE and ABORT (we can of course discuss other modes). This will lead to the introduction of transparent error handling stages in the flow.
Regardless of what mode is employed, a per-item summary should be provided in some Context object with information about what went wrong. This Context can be accessed by the caller of a pipeline.
In addition, one can also have a discussion as to how handled errors should affect Retrievables. It might make sense to include error information at a Retrievable level as well.
Currently, this is a discussion issue. I'm open for ideas and input.
Dependencies
None
Boundary Conditions
This should be implemented in a way such that the error handling logic is injected transparently when pipelines are constructed, rather than requiring the operators to manipulate the flow.
Task Description
Error handling in
vitrivr-enginecurrently has two major shortcomings.Operator's implementer decides, if an error should be handled gracefully (i.e., log and continue) or not (i.e., throw an exception). This leads to inconsistent behaviour across the a pipeline.vitrivr-engineis used as a library rather than a local service.I therefore propose three major changes to how errors should be handled:
ExtractionException. This exception reports on the error condition (retrievable, name of the operator and cause) and (optionally) wraps downstream exceptions. Throwing any other exception from within anOperatoris considered a programmer's error. Therefore, proper exception handling is needed.CONTINUEandABORT(we can of course discuss other modes). This will lead to the introduction of transparent error handling stages in the flow.Contextobject with information about what went wrong. ThisContextcan be accessed by the caller of a pipeline.In addition, one can also have a discussion as to how handled errors should affect
Retrievables. It might make sense to include error information at aRetrievablelevel as well.Currently, this is a discussion issue. I'm open for ideas and input.
Dependencies
None
Boundary Conditions
This should be implemented in a way such that the error handling logic is injected transparently when pipelines are constructed, rather than requiring the operators to manipulate the flow.