Open johnnychen94 opened 4 years ago
The coverage decreases because CI can't capture codes for InteractiveMode
. Any advice on how to test this?
I'm not satisfied with the current codes in this PR, but I don't have better ideas now.
One key question is: what advantages it brings to us if it doesn't only rely on isinteractive
for triggering the input prompt? (#8 ) The only use case I can think about is to automatically generate all reference files(but not update) in a non-interactive mode (even when isinteractive()==true
).
I like this concept overall.
I think InteractiveMode
is wrong name for this.
We have 3 behavours:
In all of them you get at least a warning and a display of the differences.
And those behavours can be applied in 2 circumstances:
We can handle the combination via multiple dispatch.
There is a case to be made for making the default setting be: Fail on Disagree, Create on Missing. That way if you knew a bunch of things had changed, you could delete all the ones you expect to be invalidated. Then rerun the tests.
Or if you expect everything to be invalidated you just switch to Create on Both.
Having used this inpractice a bit, this is fine, because one reviews the new referenced files during code-review.
Cont. #33
In this PR, a "constant" global runtime flag
TestMode
is introduced so that it's easier to handle environment variables mentioned in #8This may have many use cases, for example, in this PR I dispatch
_create_new_reference
and_update_reference
onTestMode
.The first commit doesn't change the behavior of
reference_test
. I'd love to hear advice about what you'd like to have in further commits.