Follow up surveys UX

[annikaschmid]

Icon

• [x] I think we can reuse the ‘feedback’ side app icon, because that’s only rolled out to us right now. We use the ! icon for reporting feedback / bugs

Survey list page

• [x] Would be nice to pause / stop a survey from the … menu, like you can from feature flags

Survey results page

• [x] Would like to see the actual response text, don’t care about the event name

Release conditions

• [x] I think this should be the exact same as on a feature flag at least for users in terms of functionality (e.g. having multiple condition sets, only rolling out to a % of that condition set, seeing how many users are affected)
• [x] In terms of wording / design, we can also draw inspiration from session replay filtering, e.g. we could say 'target by URL' and 'target by persons/cohorts'
• [x] I think this should be the exact same as on a feature flag at least for users in terms of styling (have a nice box, rather than the one liner we have right now)
• [x] I think it should be clearer if no condition set for users is set, will this roll out to everyone or no one? Reusing the feature flags experience solves this ("this is rolled out to xyz users")

Overview page, running survey

• [x] I would like to restart a survey, which I don’t think is possible right now
• [x] For now, it should say “question” instead of “questions”
• [ ] I think ‘question’ should have a bolder/bigger headline (like ‘payload’ on a feature flag), because it’s one of the key pieces on this page
• [ ] Question should be right above targeting, and all the ‘metadata’ like type, linked flag, start date above, that would be a more logical sorting and matches feature flags page headline hierarchy

Survey creation page

• [ ] As Raquel said, it should be clearer what the connected flag does, and how it interacts with the release conditions
• [ ] The headline size should map the feature flag creation page and the running survey page: name, description, type, linked flag = small, question and release conditions = big

Other

• [x] I like that we have a draft mode, but it's not clear at all this exists until you save the survey (I thought it goes live immediately when I tested it). Alternatively, we can reuse the same functionality as with feature flags, where we have a checkbox for ‘enable survey’ on the creation page. And then you can start / stop / restart a survey (I would prefer this, because people know this already from feature flags)
• [x] We should have a “how to implement” section on both the survey creation and the running survey screen, similar to early access / feature flags, where people can see what they have to do in order to show the survey to the user
• [x] I think we need a selector whether a survey should be distributed via site app or not. At PostHog I think we would want to use both options, but if e.g. Ben sets up a survey via API, it would still show via the site app right now?
• [ ] If I start creating a new survey and then leave the screen, there should be an 'are you sure' popup, like on insights
• [x] We should make some improvements to the empty 'results' page I am taken to when I launch the survey, it should look like it looks when I come back to this screen, with the empty list etc.

Future work

• [ ] Would be nice to have a targeted / seen / answered breakdown, so that I know ‘its working’

[liyiy]

Agree with everything! All on the todo list

I think this should be the exact same as on a feature flag at least for users in terms of styling (have a nice box, rather than the one liner we have right now)

What did you mean by this? What is "this" referring to?

[corywatilo]

I spent some time thinking through the new survey creation flow, what it could/should look like now, and how it can be extensible for future growth. So rather than starting with just designs, I started with the data structure for what a survey could look like.

First I'll cover the survey types I think we could use at launch (or a subset of these), then go over wires for the actual new survey creation flow.

Survey types

For the purposes of a near-term product launch, I went with four different templates because:

• They're relatively simple to build
• They work as a proof of concept to show the breadth of what we can build
• Users hopefully won't discount the product right off the bat because it feels somewhat substantiative

Four types of surveys

1. Feedback form (what we have today)
2. NPS/rating
3. Emoji reaction
4. Vote + feedback (a 2-step form)

These forms are limited to the fields we add to them, but you'll see how they could easily be user-modifiable in the future in the suggested json I'll provide for each.

Feedback form

Here's the json I whipped up to build it. (Note: Obviously use this as inspiration, not as an edict of exactly how to build it.)

{
  "type": "feedback",
  "title": "Share your thoughts",
  "description": "Optional form description",
  "confirmation": "Thanks! Your feedback has been sent.",
  "settings": [
    {
      "position": "right",
      "colors": [
        {
          "light": [
            {
              "primary": "rgba(0,0,0,1)",
              "background": "rgba(0,0,0,.1)",
              "border": "rgba(0,0,0,.25)",
              "buttonBg": "rgba(0,0,255,1)",
              "buttonText": "rgba(255,255,255,1)"
            },
            {
              "primary": "rgba(255,255,255,1)",
              "background": "rgba(255,255,255,.1)",
              "border": "rgba(255,255,255,.25)",
              "button": "rgba(0,0,255,1)",
              "buttonText": "rgba(255,255,255,1)"
            }
          ]
        }
      ]
    }
  ],
  "field": [
    {
      "type": "textarea",
      "label": "Feedback",
      "placeholder": "Start typing...",
      "limit": "280"
    },
    {
      "type": "submit",
      "label": "Submit"
    }
  ]
}

NPS/rating

We can combine NPS and any arbitrary number of ratings (ie: 1-5) in the same form by including some parameters for the rating field.

{
  "type": "rating",
  "title": "How satisfied are you with our product?",
  "description": "Optional form description",
  "confirmation": "Thanks for sharing!",
  "settings": [
    {
      "position": "center",
      "colors": [
        {
          "light": [
            {
              "primary": "rgba(0,0,0,1)",
              "background": "rgba(0,0,0,.1)",
              "border": "rgba(0,0,0,.25)",
              "accent": "rgba(0,0,255,1)"
            },
            {
              "primary": "rgba(255,255,255,1)",
              "background": "rgba(255,255,255,.1)",
              "border": "rgba(255,255,255,.25)",
              "accent": "rgba(0,0,255,1)"
            }
          ]
        }
      ]
    }
  ],
  "field": [
    {
      "type": "number",
      "scale": 10,
      "increment": 1
    }
  ]
}

You'll see in the field, we specify the scale and increment of numbers that show. (We don't have to expose this to a user - we could easily create an NPS form, which includes 10, and a generic rating form, which includes 5 - and largely use the same code to power it.)

Emoji reaction

{
  "type": "emoji",
  "title": "How do you feel about this page?",
  "description": "Optional form description",
  "confirmation": "Thanks for your feedback!",
  "settings": [
    {
      "position": "right",
      "colors": [
        {
          "light": [
            {
              "primary": "rgba(0,0,0,1)",
              "background": "rgba(0,0,0,.1)",
              "border": "rgba(0,0,0,.25)"
            },
            {
              "primary": "rgba(255,255,255,1)",
              "background": "rgba(255,255,255,.1)",
              "border": "rgba(255,255,255,.25)"
            }
          ]
        }
      ]
    }
  ],
  "field": [
    {
      "type": "emoji",
      "label": "😡"
    },
    {
      "type": "emoji",
      "label": "😑"
    },
    {
      "type": "emoji",
      "label": "😃"
    }
  ]
}

In the above, we'd have to specify the order (low to high) so we know which emoji refers to what, in the same way the scale (1-5, 1-10) would work the same way.

Vote + feedback

This is a multi-screen survey where you have to answer the first question to get to the second question.

This introduces the concept of pages that could (in the future) be re-ordered.

{
  "type": "voteFeedback",
  "title": "How do you feel about this page?",
  "description": "Optional form description",
  "confirmation": "Thanks! Your feedback has been sent.",
  "settings": [
    {
      "position": "right",
      "colors": [
        {
          "light": [
            {
              "primary": "rgba(0,0,0,1)",
              "background": "rgba(0,0,0,.1)",
              "border": "rgba(0,0,0,.25)",
              "buttonBg": "rgba(0,0,255,1)",
              "buttonText": "rgba(255,255,255,1)"
            },
            {
              "primary": "rgba(255,255,255,1)",
              "background": "rgba(255,255,255,.1)",
              "border": "rgba(255,255,255,.25)",
              "button": "rgba(0,0,255,1)",
              "buttonText": "rgba(255,255,255,1)"
            }
          ]
        }
      ]
    }
  ],
  "pages": [
    {
      "field": [
        {
          "type": "vote",
          "options": [
            {
              "value": false,
              "icon": "thumbsUp"
            },
            {
              "value": true,
              "icon": "thumbsDown"
            }
          ]
        }
      ]
    },
    {
      "field": [
        {
          "type": "textarea",
          "label": "Any additional feedback?",
          "placeholder": "Start typing...",
          "limit": "280"
        },
        {
          "type": "submit",
          "label": "Submit"
        }
      ]
    }
  ]
}

Setting up an array of pages gets us to a spot where a) we can easily build multi-page forms, and b) they can eventually be user-customizable.

New survey setup flow

Rather than do a multi-page UI (which would be cleaner), I kept everything on one page here, with a live preview of what the widget I've chosen looks like in the place it would appear.

It breaks the steps into:

1. Form type
2. Presentation
3. Form fields
4. Customization
5. Targeting

Form type & Presentation

In the above wire, I went for a more visual approach here (showing previews of what you'd get). For a barebones MVP, we could simplify it by just doing standard form fields like seen below, though it really wouldn't take much to get it to the place outlined above, which is much more clear to a user.

Customization

We could probably simplify this (ie: define a text color, an accent color, and a background color) but ultimately I think having more customization available is better, especially for a developer product.

@smallbrownbike and I chatted about this and we'd be down to work together on building out the front end templates/widgets since we have some relevant experience here with theming, shadow DOM, CSS variables, etc. So if y'all only want to focus on the UI of building/editing/viewing results, we'd be happy to work on the part that gets embedded.

Targeting

This is the most updated portion of the existing MVP, as it moves feature flags in here, as well as makes it a little more explicit about how you can target (especially with labels).

---

The above would get us to a really solid v1. Up to you how much of this we build today, but I think this would put us in a good spot to make sure we're building for the future without overbuilding right now.

[annikaschmid]

Thanks Cory, this is great!

Here is some feedback from my side:

Survey types

+1 that this is a great set and structure to start with. I think Li is already working on some of those. I also really like the visual approach (as compared to the dropdown).

I have a recommendation for one more, which we already launched: We have a new survey type called 'link' aka the user interviewer, which is essentially customisable text, with a button that goes to a third-party link. Could you add this to your options? I saw some other tools have an option on the button to do this (see screenshot). Another idea from James was to show "user interview" as a "form type", so that we guide users more to use this to set up user interviews, which I think is a great idea.

New survey setup flow

• For "presentation", how would the preview look like if I chose API or modal? Is it still previewed as the the popover in the corner?
• The feedback input label I would make optional as well
• "Thank you" screen: This means users would have to manually dismiss the popup. Can we auto-close it after a couple of seconds, like the error toasts? That way, the "popup dismissed" event lets you track accurately who is not interested in filling out the survey

Customization

If you and Eli want to work on the site app / popup / modal styling options, that would be awesome. I think those have a lot of potential. I also think the default option we have right now could look a bit prettier. We also have the early access site app, which maybe once we have this done for surveys, we can replicate it for early access.

Targeting

I wonder if this area should look "bigger", because we have similar targeting on flags ("release conditions") and there it looks much more impressive 😅

• The "all users" radio button is very useful
• I think we should have AND/OR operator between the different options so that it's clearer how it behaves
• For "match user properties", on feature flags, people can set a percentage and then see an estimate of how many people this will be rolled out to (screenshot). We want to add this to surveys as well. Could you maybe add this to your wires?

Other

When a user selects API/Headless, would you want to show the implementation guide on the survey setup screen? Or do you think it's fine if we only show it when the survey is set up? Right now, we do the second, but I wonder if that leaves users confused about how to actually turn it on / make it work.

Another thing we will add soon is a "recontact period", which can be set on a global level for surveys. Sprig has this option where you can overwrite it on the survey itself. I don't think this is critical, but I think the global setting is important, so that users don't get spammed. Adding it here for inspiration and that we keep thinking about it :)

[liyiy]

Thanks for the feedback Cory! I agree that it's important to make sure the survey popup types are functional (working on more basic types right now). I was wondering if you had any additional feedback about how the survey results + view pages look like currently and what are the best ways to summarize an "overview" of the survey created, for example.

Other points of response:

@smallbrownbike and I chatted about this and we'd be down to work together on building out the front end templates/widgets since we have some relevant experience here with theming, shadow DOM, CSS variables, etc. So if y'all only want to focus on the UI of building/editing/viewing results, we'd be happy to work on the part that gets embedded.

This sounds great. Surveys functionality for response collection and events capture are integrated along with the UI template on the plugin app currently so there's a bit of work to do in order to separate function from form, and would love to work together on this!

So rather than starting with just designs, I started with the data structure for what a survey could look like.

Sorry I didn't include this anywhere earlier, here's a bit of background on how the data model currently works for surveys:

Surveys:

1. type: (popover, permanent button, etc)
2. linked_flag: feature flag to be associated with the survey as part of feature success
3. targeting_flag: how user properties are stored like feature flag filters
4. conditions: This is where we store things like url/selector conditions, it's an open JSON field right now so we can continue to add conditions along the way. { url: 'posthog', selector: null, ... } Both targeting_flag and conditions let the user choose who/where to target the surveys to
5. appearance: This is where we store background color, button color, text color, etc. Open JSON field right now so we can continue adding on. However we should be careful on how much optionality we allow as at some point it sense for the user to do it the headless API route instead.
6. questions: Questions are stored like this: open text example: [{ type: 'open', question: 'what do you think?' }] multiple choice example: [{ type: 'multiple_choice', question: 'pick one of these', choices: ['choice 1', 'choice 2', ...] }] NPS/ratings example: [{ type: 'rating', rating_display: 'numbers', scale: 5, question: '' }]

Surveys that are created are then evaluated by the posthog-js library, where it determines which surveys to show the user, before it's sent to the feature-surveys plugin app. There, the actual survey UI is created based on the configuration of appearance and questions fields. And then whenever a user is shown a survey, dismisses it, or completes it, we send in the associated events to PostHog. Let me know if this makes it clearer how the system is currently structured~

[corywatilo]

I was wondering if you had any additional feedback about how the survey results + view pages look like currently and what are the best ways to summarize an "overview" of the survey created, for example.

Sorry I haven't had a chance to wire this up yet. I think in the short term, a few small changes would clean it up nicely.

• It might just make sense to use some dashboard elements in the survey results view. For example, instead of having Survey shown and Survey dismissed columns, maybe a pie chart (or bar?) that shows:

  • Total seen/shown (whole pie)
  • Responses
  • Survey dismissed

• If possible, it would also be good to remove the Select an event option and the selected survey ID (since that's the basis of the result data and I don't imagine this would ever get changed?).

• I'd remove the DB icon in the headers of each box.

• Parse for line breaks (vs displaying \n)