Don't rely on AssertionError from failure

[johlrogge]

Both attr and structure rely on assertions throwing exceptions. This is of course not correct behavior since the exception-throwing part can be configured in referee. It will work for us for a while longer while we have focus elsewhere but we will have to fix this when we are production ready.

[meisl]

Yep, thx for making it an issue. I've also added a deferred test with a name stating this TODO to make really sure we won't forget ;)

I'd like to volunteer for this one, but of course only after the longer while

...and I'd suggest to change the title into "Don't rely on AssertionError from failure" or so.

[johlrogge]

Feel free! See my comment in #6 for some input

[meisl]

Ok, started working on applying my approach from ramp-resources tests, as said in #10. And, as promised, trying to stay open for plugging in / replacing some parts by @johlrogge's 'internal' / 'raw' assertion results.

While making it a bit more concrete I came to think, fortunately, that actually there shouldn't be too much overlap. My "trick" from ramp-resources - basically: intercepting fail - is only a rather small and basic thing and layers around it are needed to make it reasonably usable. So that means it will be this (and not much more, if anything at all) that will be replaced by (or flanked by, as it may turn out) 'raw' assertion results.

In the following I'll be using the terms/definitions from #11, they'll appear in _bold italic_ - even though it looks pretty ugly...:( I'll adjust my description to whatever we decide to change wrt there (within a reasonable time frame).

So here's the agenda:

• As the first / innermost layer I'll make one thing to represent all aspects of an _OLA_s _outcome_. It won't be a promise at first, just something. But ONE thing. This way it should be easy to adapt / exchange.
• Then I'll make the _meta assertion_ fails (there are more but that's the one I'll focus on), just like any other custom assertion. That is: start with an _assertion definition_ as usual, ( _implementation_ based on stuff from ramp-resources tests) and test that directly ( _implementation_ is _FUT_, just as you @johlrogge do in #10, "test from outside"). Note that this is NOT to say that buster.assert.fails should be publicly available. It is just the route for now as I think it will be interesting to see how far I get with treating _meta_ stuff just as normal, and what will be the walls I'm going to bump into.
• Next, it seems pretty clear to me that _meta assertions_ are not really something to be used at _user-level_, not even in our internal tests. They're just not abstract enough. The current test helper methods assertionTests in referee and makeTests here are indicators for this. So it is such helper methods that will use assert.fails directly but in _user-level_ tests - including our own - we'll only use those helper methods to test assertions. So this step is about what the essential building blocks of such helper methods are. The basic scheme of providing callbacks such as pass, fail as currently in makeTests looks like one, although it might need to be refined.
• Right now I'm working on a test helper method parametricTests, something I was always wishing for and now - in _meta_ land - really need. The current makeTests can be based on that, so there's a bit of progress regarding generalization of testing assertions (maybe sometime improving referee tests). I'll put up a pr with parametricTests for referee-combinators/develop soon.
• One particular thing (but of course not the only one) that I'm trying to address with parametricTests is #7, _OLA_ and _FUT_ coming from the same referee or not.
• So far, I've been doing all this only in referee-combinators' test/test-helper.js but it's pretty clear that a) I'll have to make some adjustments in referee soon and b) referee-combinators/tests/test-helpers isn't really the place where it should live. So we'll have to define a common environment soon. That is: where to have the clone of referee & (a reliable recipee for) how to get it there. Also, should I fork your fork of referee or fork referee directly? Probably the latter but I'm not sure. Currently I have the fails stuff on a branch "meta-assertions" of referee-combinators.

[meisl]

Re environment (last bullet): what's described in the README, is that working for you? That's what I'm doing right now.