search

presleyp / Speriment

A program to make it easier to write online experiments. Speriment takes a simple description of your experiment in Python and uses it to run online experiments with the use of PsiTurk and Mechanical Turk.
GNU General Public License v2.0
6 stars 1 forks source link

Let OuterBlocks have criterion #22

Closed presleyp closed 9 years ago

presleyp commented 9 years ago

This would be easy to do but I have to think about how it should work. If InnerBlocks don't have criterion, fine. If they do, does it calculate only over their last iteration?

presleyp commented 9 years ago

It's also fine if only one inner block has a criterion. Just a way of bundling training and testing.

If multiple layers have criteria, I think I'll only count the last iteration. I can't imagine wanting to grade over the whole history, and this is a nice safe way to layer criteria.

presleyp commented 9 years ago

New question: inner blocks reorder questions when looping. Should outer blocks reorder exchangeable blocks when looping? This would not apply to counterbalanced blocks, so I'm tempted to say no so that they work the same except for determinism. But it may come out later than people use exchangeability in a way that makes the reordering beneficial.