Clarify parameters on "New experiment" page

[stsievert]

On the NEXT frontend GUI, Dashboard > Select project > new experiment > * presents a bunch of options which are not clearly explained. At first glance, it's not clear what "context type" means or what impact it has.

I'm thinking of having an interface like below:

The entire page needs clarification: from context to algorithm choice.

[stsievert]

I have addressed this issue by adding little question mark buttons. These button have an onclick element that calls some javascript that makes a modal display the helper text.

Screenshots

Screenshot:

When the question mark is clicked, a little modal pops up:

Questions on parameters

I wasn't completely sure on some parameters.

• General
  • [x] What wording should I use with 'confidence level'?
  • [x] What does each option do for "algorithm management"? The current text is some dummy text and not very helpful
  • [x] What does each option do for "participant to algorithm" management? The current text is some dummy text and not very helpful
• Dueling Bandits/Tuple
  • [x] What does the parameter context-type and context-text/image mean? Where is it displayed? I ran a quick test and didn't see it anywhere.
  • [x] When adding an algorithm, what the algorithm BR_LilUCB do? What does it stand for?
• Triplet
  • [ ] What does the Crowd Kernel sampling algorithm do? A high level description?

@lalitkumarj, can you answer these questions?

Clean up

I still have a couple things to clean up:

• [ ] Document the fact that Algorithm Proportion adds up to 1, and probably enforce it

[kgjamieson]

Confidence level: "Many algorithms provide statistical guarantees. A confidence level of alpha indicates that with probability at least 1-alpha the algorithm will eventually converge to the underlying truth, assuming standard independent and identically distributed data assumptions. A typical value is 0.05."

Algorithm mangagement: "Participant-to-Algorithm Management If multiple algorithms are used in an experiment (or there is a test set as in triplets) this option dictates how participants are assigned to algorithms.

One-to-many Each query generated for a participant is randomly and independently chosen according to the fixed proportions. Consequently, the queries asked of the participant over the course of the session are generated from many algorithms.

One-to-one The first query generated for a participant is randomly and independently chosen according to the fixed proportions. Every query thereafter is generated by the same algorithm. Consequently, the queries asked of the participant over the course of the session are generated from a single algorithm."