search

gnecula / bond

Spy-based testing library.
http://necula01.github.io/bond/
Other
7 stars 3 forks source link

ease of passing tests if you change the observation directory #8

Closed xkrogen closed 9 years ago

xkrogen commented 9 years ago

Right now if you change the observation directory, on the next run, all of your tests will pass without warning or anything. It would be really easy to accidentally change the observation directory, break a bunch of things, run the tests and have them succeed, and then check in that code. It's kind of an obscure situation but I kind of just almost did it to myself (because I was fiddling around with the code that sets the observation directory, broke it, and all of my tests were passing).

Not sure if we need to do anything about this (beyond maybe a comment in the documentation), but I think it's something to consider.

gnecula commented 9 years ago

Maybe it makes sense to consider that the first time you reconcile changes the reference file is empty. This forces you to look at the first saved observation, as you should. And it will also have the additional effect of signaling when you changed the observation directory.

xkrogen commented 9 years ago

Yeah, I agree with this. You should need to manually verify the results the first time, it doesn't really make sense that it automatically assumes the first run is correct.

On Wed, Oct 14, 2015 at 6:12 AM, George Necula notifications@github.com wrote:

Maybe it makes sense to consider that the first time you reconcile changes the reference file is empty. This forces you to look at the first saved observation, as you should. And it will also have the additional effect of signaling when you changed the observation directory.

— Reply to this email directly or view it on GitHub https://github.com/necula01/bond/issues/8#issuecomment-148044865.

gnecula commented 9 years ago

I have fixed this for pybond.

xkrogen commented 9 years ago

Fixed in rbond as well.