In a test suite I am working on, we are testing both "nominal cases" ("happy path") and "attack cases" (non-happy-path, where we expect script failure).
The attack cases require triggering specific failures, and those failures must emit error codes into the logs. We parse the logs, and if we don't find the specific failure code we expected to trigger, then the test fails.
We are taking this approach because without matching on error codes, the alwaysFail validator (for instance) would pass every attack case. This clearly doesn't mean that the alwaysFail validator matches the specification of some foo script.
Taking this approach is complicated by the presence of functions that are partial, but either fail silently (emitting no logs) or fail with a hard-coded error message, since we can't emit a particular error code.
I would like to see tracing versions of partial functions that allow us to log a custom error code at a specified logging level.
We'll be writing versions for this project on an as-needed basis -- specifically, when we realize we can't trigger a specific error code. @kozross's work on the PTryFrom update is such a case -- guidelines on naming or other conventions are appreciated.
In a test suite I am working on, we are testing both "nominal cases" ("happy path") and "attack cases" (non-happy-path, where we expect script failure).
The attack cases require triggering specific failures, and those failures must emit error codes into the logs. We parse the logs, and if we don't find the specific failure code we expected to trigger, then the test fails.
We are taking this approach because without matching on error codes, the
alwaysFailvalidator (for instance) would pass every attack case. This clearly doesn't mean that thealwaysFailvalidator matches the specification of somefooscript.Taking this approach is complicated by the presence of functions that are partial, but either fail silently (emitting no logs) or fail with a hard-coded error message, since we can't emit a particular error code.
I would like to see tracing versions of partial functions that allow us to log a custom error code at a specified logging level.
We'll be writing versions for this project on an as-needed basis -- specifically, when we realize we can't trigger a specific error code. @kozross's work on the PTryFrom update is such a case -- guidelines on naming or other conventions are appreciated.