Consider using Distribution Viewer to support single pref-flip experiments

[gregglind]

Single pref-flips experiments will create new populations to compare. Claim: D-V might be the right place to do this.

This might entail:

• more populations in the api.
• some nice way of 'prioritizing' certain metrics (in the display)
• big stretch goal: support for inference?

[openjck]

Is D-V Distribution Viewer?

[rafrombrc]

Yes, D-V in this case means distribution viewer. Note, however, that while this may mean the existing dist viewer instance that is deployed, another (more likely?) possibility is that it's a different installation of the same underlying code, connected to a different data set.

[gregglind]

The vagueness here is intentional. There is a NEED to dashboard collected Histogram data single-pref-flip experiments. Even better if it has stats (narrative), lots of check marks, etc.

Whatever we decide below my inform who should do the work (or the reverse!).

I see some things that could meet this requirement:

• changing the existing instance (Gauss?) to expose those "populations"
• 'cloning' the code base
• cloning just the D3 parts
• making some single-page app that does this using a similar data api

I love these things:

• it exists!
• query args handle pops, timestamps
• comparisons / overlays
• histograms plotting
• api / server for populating the points, api.md

Things that seem to be missing / could grow / awesome ++

• 'narrative' boxes that summarize common tests (building on @rvitillo permutation / chi-square code from e10s work). DID IT CHANGE? IF SO, HOW
• different aesthetics
• an 'even more tl;dr view' that shows only charts with changes, etc

GL

On Tue, Mar 14, 2017 at 3:38 PM, Rob Miller notifications@github.com wrote:

Yes, D-V in this case means distribution viewer. Note, however, that while this may mean the existing dist viewer instance that is deployed, another (more likely?) possibility is that it's a different installation of the same underlying code, connected to a different data set.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mozilla/distribution-viewer/issues/217#issuecomment-286552221, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKAj8aIL9qTbOvZHvg4_EYMGC2iNHdQks5rlvrRgaJpZM4MbUdV .