amazon-ion / ion-c

A C implementation of Amazon Ion.
https://amazon-ion.github.io/ion-docs/
Apache License 2.0
166 stars 43 forks source link

ion-test-driver workflow will fail whenever lines are added/removed from a file that produces expected errors. #317

Open tgregg opened 1 year ago

tgregg commented 1 year ago

This happens because the failure message includes the line number from which the error originated. Example here. This requires developers to manually look through the test-driver results and determine that they are false positives, introducing the risk of an incorrect evaluation that results in a behavioral change being allowed through. We should consider whether the best solution is to build ion-c in release mode (instead of debug mode) in ion-test-driver, or to change the evaluation logic in ion-test-driver, or something else.