the names 'condition' and 'counterbalance' can be confusing

[isofer]

I think the current naming conventions can be a little confusing. The 'condition' and 'counterbalance' parameters are essentially both between-subjects factor, so it might be better to use names that reflect this.

condition can be replace with level_a (The level is factor A), and counterbalance with level_b (the level in factor B).

Of course, changing names will create compatibility issues, so it might be best to wait until several compatibility breaking issues will occur. On the other hand, psiTurk is very young, so it is easier to deal with these issues while the community of users is small.

[johnmcdonnell]

The intention was that manipulations of theoretical interest would be coded in cond and those that were just designed for counterbalancing would go in counterbalance. I understand though that the difference is conceptual and may be confusing.

Should we maybe just drop counterbalance altogether? Or write documentation saying that if it doesn't make sense to you then just don't use it?

[isofer]

I think it's great that psiTurk support two factors, so there is no reason to drop counterbalance. It would allow users greater flexibility. if a user wants to do two-way anova, or to collapse over everything, there is no reason why he shouldn't. My view is that the documentation should say something like "for historic reasons, one is named condition, and one is counterbalance".

If you decide to change the names. Then maybe consider a single array variable - n_levels. len(n_levels) would be the number of factors, and n_levels[i] would be the number of levels in factor i. This would allow unlimited number of factors.

But I'm fairly new to HTML/JS/PHP, so I don't know if that's easy to implement.

On Mon, Nov 18, 2013 at 5:56 PM, John McDonnell notifications@github.comwrote:

The intention was that manipulations of theoretical interest would be coded in cond and those that were just designed for counterbalancing would go in counterbalance. I understand though that the difference is conceptual and may be confusing.

Should we maybe just drop counterbalance altogether? Or write documentation saying that if it doesn't make sense to you then just don't use it?

— Reply to this email directly or view it on GitHubhttps://github.com/NYUCCL/psiTurk/issues/53#issuecomment-28747013 .

[gureckis]

I do think it would be nice if this was more flexible. For example there are multi-way factorial designs (suggested by @isofer) where N factors are counterbalanced. In addition, one might care that factor 1 is evenly balanced in a fixed sample (e.g., 100 subjects, 25 in each of four conditions) but that factor 2 is simply non-repeating (it has 127 levels, say, and you just want to not sample those in a biased way, even though you don't expect to experience all of them). This can be done currently, but not in a way that ensure the "important" factor is more evenly balanced than the "less" important factor.

In general, I do think a generic interface for specifying multiple types of designs would be a useful extension to the cond/counterbalance thing. We would happily receive a pull request along those lines.