search

TuringLang / docs

Documentation and tutorials for the Turing language
https://turinglang.org/docs/
MIT License
231 stars 99 forks source link

Add assertion to tutorial 2 #248

Closed rikhuijzer closed 3 years ago

rikhuijzer commented 3 years ago

This is a suggestion on how to verify the tutorials. Would you say that

if the student parameter of the chain for tutorial 2 is within -3 < parameter < -1 and no errors were reported during build, that then the tutorial is most likely valid?

So, if there are plotting errors or whatever, that will be caught by general errors. This assertion is added to ensure that the chain converges to the right spot. If the parameter is not in this range, something is probably wrong.

If this is a reasonable assumption, then we can start adding these kinds of assertions to all tutorials, setup CompatHelper with a private key so that it runs CI and, finally, we can merge CompatHelper PRs very quickly!

rikhuijzer commented 3 years ago

@devmotion, @cpfiffer or @torfjelde, is this how you would verify chains?

torfjelde commented 3 years ago

I like the idea of having verifications in here, for sure! I general, just asserting that a value lies within a range is sufficient though, so I think he would have to be determined on a case-by-case basis.

For the particular case you're suggesting, and if you're talking about the median, then yes this seems reasonable. You could also add a check for the predictons at the end too, e.g. ensure that the accuracy is >0.6 or something.

rikhuijzer commented 3 years ago

@torfjelde thank you for taking a look. Could you verify if this is now what you meant and merge if you agree?

I've also bumped the Turing version now and linked the relevant CompatHelper PR.