[RFC][Draft] First-time User Experience

[pringshia]

I think the onboarding experience for Mephisto will be really important in making sure this product sticks and that we maintain a high retention rate.

This RFC is an attempt to gather some initial thoughts on optimizing for the NUX (new user experience) aka first-time user experience and align on proposed action items.

I pose some questions and some answers below, the latter of which is not meant to be exhaustive.

1. The Installation Experience

The question

What should be the method of delivery for this project?

Some answers

• Option 1: The user can download a package, e.g. pip install mephisto which can give them access to the interface/tool to use (via CLI command).
  • In this case upstream updates can be incorporated by downloading the updated version of the python package.
• Option 2: The user can clone the repo and use that to access the tools provided within the repository (e.g. similar to projects like ParlAI).
  • In this case upstream updates must be managed by pulling from the git repo periodically. If the user has made some changes, they must manage the merge conflicts.

2. The Quick-Start Experience

The question

How can we empower users to get immediate value? How can we make the user feel empowered and accomplished with the tool, with minimal overhead? How can we dazzle the user with the product's potential?

Some answers

• Have a task gallery available that shows common templates that already exist, selling on: ease of creating new templates, tools around inspecting templates, diversity of templates (this last one will grow over time and will be a long-term payoff).
• Create a quick-start guide with a dead-simple use-case, e.g. start a static task locally using a predefined task template, and gather a result.
• Provide an incrementally more advanced use-case of having a dynamic task.
• Take a Documentation-Driven Development approach to ensure that we are mindful of the learning curve of features as we build them out.

3. Nailing Down the Touchpoints

The question

What are different opportunities for users to formulate a perception about our product? How can we make sure we win in each of those areas?

Some answers

1. Github README.md
2. Website?
3. Quick Start Guide
4. Documentation
5. Github issues

4. The API Experience

The question

How can we ensure that we are being intentional & thoughtful about API design decisions?

Some answers

• Reference existing style guides & best practices:
  • https://devcenter.heroku.com/articles/cli-style-guide
  • https://github.com/cloudfoundry/cli/wiki/CF-CLI-Style-Guide

[JackUrb]

I'm thinking the optimal approach is a slightly more automated version of what we had working in visdom - namely that the master branch can be installed via pip install -e . but some stable branch can be used from pip install mephisto

I think though that unless the cli is able to set up directory structures and such for the user's special cases, using the defaults that one can put together by cloning mephisto is slightly more optimal.

[pringshia]

Threw together a splash page:

[JackUrb]

Coming back around to this, we've got a lot together on this already, and may have just a little bit left to do:

• pip install mephisto from PyPI and pip install -e . from source both work
• Website has quickstart and tutorials

I suppose we could revisit the README.md here if we think it's valuable having documentation on the github (and not the site) be strong.

One thing we could definitely use though is templates for PRs and Issues