When running Fuchu tests via FSI, failing Unquote assertions do not throw an exception

[daniel-chambers]

We're currently writing tests for our .fsx scripts by writing Fuchu tests inside another .fsx that loads in the script under test; we then run the test .fsx via FSI during our build. Unfortunately, Unquote assertions (such as =!) detect that the assertion function is used inside FSI and stop throwing exceptions, instead preferring to print output directly to the console. This is causing our failing tests to not actually fail.

The offending check is done here.

I think that it would be a good compromise to change that code so that it tries to detect the presence of a test framework first, then falls back to detecting FSI. In my opinion, it's unlikely that those using Unquote in FSI to do interactive comparisons will also have a test framework assembly loaded in their App Domain.

What do you think?

[stephen-swensen]

Yes, I think there is good sense to your recommendation: Fuchu is unique among most test frameworks in that it allows programmatically executing tests, thus changing the original assumptions Unquote made about output preferences.

The diff for your pull request looks solid too at first glance (thank you!). I'm going to test it locally and think on it a bit more, but I am strongly favoring adopting it at this point.

[s-trooper]

i have the same issue, is there any schedule for this ?

[stephen-swensen]

I am aiming to have a release out with the fix for this within the next week.