Question: error path handling

[Radamonas]

ETLs like SSIS supports error handling by calling/executing routines to handle failures:

Also there is Event handler for all errors, like in pic below:

Is there ways to do similar routines with Etl.Net?

[paillave]

error handling in SSIS is done at controlflow level. But it has nothing for the datafow. If the dataflow, the control flow handles it, and take the proper decision. ETL.NET is the replacement of the dataflow, and the external dotnet code that triggers it is the replacement of the control flow. If any failure, the process will fail, then the part that triggered the failed ETL.NET process will get all the details about the failure in the response.

This said, you can handle exception your self within select or do operators and return a payload that permits you to distinguish rows failed from rows that succeeded so that you can apply a different mass process on it.

[mickvikt]

It is clear about error handling in select and do operators, but what is the idiomatic way for handling errors/exceptions in other operators like CrossApplySqlServerQuery or WriteToFile.

Should the whole chained statement be wrapped in try/catch?

[paillave]

like in SSIS or many other ETL, in this situation, your whole process will fail, and you will have to catch it from outside control flow like described here: https://paillave.github.io/Etl.Net/docs/tutorials/trackAndCheck#get-the-error-if-it-occurs.

The detailed behavior of the engine is described here: https://paillave.github.io/Etl.Net/docs/quickstart/principle#detailed-principle

• When an error occurs in the operator, this one emits it to the runtime.
• When the runtime is notified of an error, it requests every operator to stop its job, then stops, and then returns a failed execution status along with the notified error.

[mickvikt]

Thank you, @paillave