Currently orca_test uses an assertion model, where a failing check raises an exception and causes code execution to stop. @sablanchard has a use case where it would be better to print a report of all the failures at once. This is similar to issue #5.
(Raising an exception makes sense if you're using orca_test to check data dynamically within a simulation; printing a report makes sense for things like one-off data validation.)
Immediate solution
We realized you can get this behavior without changing orca_test, though, using an approach like this:
import orca_test
from orca_test import OrcaAssertionError
specs = [...] # list OrcaSpecs here
problems = False
for spec in specs:
try:
orca_test.assert_orca_spec(spec)
except OrcaAssertionError as e:
problems = True
print(str(e))
pass
if problems:
raise OrcaAssertionError("Problems found")
Here's a self-contained, fully-functional script demonstrating this usage: orca_test_demo.py
And this is what the output looks like:
Table 'households' is not registered
Table 'buildings' is already registered
Table 'badtable' is registered but cannot be generated
Column 'index' is not registered in table 'buildings'
Column 'price1' is already registered in table 'buildings'
Column 'badcol' is registered but cannot be generated
Column 'price1' is not set as the index of table 'buildings'
Column 'strings' has type 'object' (not numeric)
Column 'price2' is 20% missing, above limit of 0%
Column 'price1' has maximum value of 50, not 25
Column 'price1' has minimum value of -1, not 0
Column 'price2' is 20% missing, above limit of 10%
Column 'fkey_bad' has values that are not in 'zone_id'
Injectable 'nonexistent' is not registered
Injectable 'rate' is already registered
Injectable 'bad_inj' is registered but cannot be evaluated
Injectable 'dict' has type 'dict' (not numeric)
Injectable 'rate' has value of 0.64, less than 5
Injectable 'rate' has value of 0.56, greater than -5
Injectable 'rate' is not a dict
Injectable 'dict' does not have key 'Oakland'
Traceback (most recent call last):
File "orca_test_demo.py", line 124, in <module>
raise OrcaAssertionError("Problems found")
OrcaAssertionError: Problems found
Longer-term solution
Ultimately, though, we probably want to add multiple modes to orca_test, perhaps as global settings.
Currently
orca_testuses an assertion model, where a failing check raises an exception and causes code execution to stop. @sablanchard has a use case where it would be better to print a report of all the failures at once. This is similar to issue #5.(Raising an exception makes sense if you're using
orca_testto check data dynamically within a simulation; printing a report makes sense for things like one-off data validation.)Immediate solution
We realized you can get this behavior without changing
orca_test, though, using an approach like this:Here's a self-contained, fully-functional script demonstrating this usage: orca_test_demo.py
And this is what the output looks like:
Longer-term solution
Ultimately, though, we probably want to add multiple modes to
orca_test, perhaps as global settings.