Fragility in Handling "Expected Exception" tests

[arcfide]

The current approach to handling expected exception tests is to compare them against the result obtained from ⎕DM. Specifically:

Z←(↑⎕DM)1
:If exception≢⍬
  expect←exception
  Z[2]←0
  t←⎕TS-t
:EndIf

There's an issue here. ↑⎕DM will be a matrix containing line information and so forth. This is used when doing the reporting to give more information about where the error has occurred inside of the test, and helps when you are not dealing with expected exception tests. However, if you do expect an exception, it's not a good idea to require that the entire line information be included as a part of the expected error. Instead, either the short error message 0⊃⎕DM or the actual trapped error signal should suffice. This improves the ease of writing these tests, and makes them more robust against changes in source code which do not actually change the substantive behavior of the program.

I am not sure what the best way to handle this is.

[Gianfrancoalongi]

Hi Aaron! Thanks for the insight, I agree that the current way is clumsy and expects too much information - and I would love to explore a better approach. What if it would be based on the programmer specifying either of short error message or the code and the system checks for both?

On Wednesday, July 16, 2014, Aaron W. Hsu notifications@github.com wrote:

The current approach to handling expected exception tests is to compare them against the result obtained from ⎕DM. Specifically:

Z←(↑⎕DM)1 :If exception≢⍬ expect←exception Z[2]←0 t←⎕TS-t :EndIf

There's an issue here. ↑⎕DM will be a matrix containing line information and so forth. This is used when doing the reporting to give more information about where the error has occurred inside of the test, and helps when you are not dealing with expected exception tests. However, if you do expect an exception, it's not a good idea to require that the entire line information be included as a part of the expected error. Instead, either the short error message 0⊃⎕DM or the actual trapped error signal should suffice. This improves the ease of writing these tests, and makes them more robust against changes in source code which do not actually change the substantive behavior of the program.

I am not sure what the best way to handle this is.

— Reply to this email directly or view it on GitHub https://github.com/Gianfrancoalongi/APLUnit/issues/19.

Sent from Gmail Mobile

[arcfide]

I think the salient information is the exception number associated with the error. Depending on the type and location of the error, the message will not always be the same. Thus, I think it would be best to capture two bits of information, the actual exception code thrown, and the complete ⎕DM information. The one should be used for comparing the exception value, and the other for printing diagnostic information.

On Thu, 2014-07-17 at 23:36 -0700, Gianfranco Alongi wrote:

Hi Aaron! Thanks for the insight, I agree that the current way is clumsy and expects too much information - and I would love to explore a better approach. What if it would be based on the programmer specifying either of short error message or the code and the system checks for both?

On Wednesday, July 16, 2014, Aaron W. Hsu notifications@github.com wrote:

The current approach to handling expected exception tests is to compare them against the result obtained from ⎕DM. Specifically:

Z←(↑⎕DM)1 :If exception≢⍬ expect←exception Z[2]←0 t←⎕TS-t :EndIf

There's an issue here. ↑⎕DM will be a matrix containing line information and so forth. This is used when doing the reporting to give more information about where the error has occurred inside of the test, and helps when you are not dealing with expected exception tests. However, if you do expect an exception, it's not a good idea to require that the entire line information be included as a part of the expected error. Instead, either the short error message 0⊃⎕DM or the actual trapped error signal should suffice. This improves the ease of writing these tests, and makes them more robust against changes in source code which do not actually change the substantive behavior of the program.

I am not sure what the best way to handle this is.

— Reply to this email directly or view it on GitHub https://github.com/Gianfrancoalongi/APLUnit/issues/19.

Aaron W. Hsu | arcfide@sacrideo.us | http://www.sacrideo.us Please support my work: https://www.gittip.com/arcfide/

לֵ֤ב חֲכָמִים֙ בְּבֵ֣ית אֵ֔בֶל וְלֵ֥ב כְּסִילִ֖ים בְּבֵ֥ית שִׂמְחָֽה׃