Free-form surveys

[nickbadams]

Expected behaviour

When in Timeline view and clicking on a post title in order to view post responses, we would like to see the responses displayed in the same sequence as the original survey questions.

Actual behaviour

The responses display in order of question type.

Steps to reproduce the behaviour/error

See DemoWatch deployment's posts. Survey Tasks 1, 2, and 4 display according to the bug. 3 displays properly (for some reason).

Aha! Link: https://ushahiditeam.aha.io/features/PROD-681

[jshorland]

@brandonrosage I'm not sure I completely understand this. Is this clear to you?

[brandonrosage]

@jshorland After investigating the issue on demowatch.ushahidi.io, I understand the problem. I'll carve out time to address this this week at the hit team.

We hear you, @nickbadams. I'll be in touch shortly.

[nickbadams]

Y'all are great. Thank you!

On Sun, Aug 21, 2016 at 7:43 PM, Brandon Rosage notifications@github.com wrote:

@jshorland https://github.com/jshorland After investigating the issue on demowatch.ushahidi.io, I understand the problem. I'll carve out time to address this this week at the hit team.

We hear you, @nickbadams https://github.com/nickbadams. I'll be in touch shortly.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ushahidi/platform/issues/1335#issuecomment-241303581, or mute the thread https://github.com/notifications/unsubscribe-auth/AFfl9UgG8P1OJUy1KvPfrZQZD4B5TmuYks5qiQy4gaJpZM4Jhv_t .

Nick Adams, Ph.D.Research Fellow – Social Science,

Berkeley Institute for Data Science University of California, Berkeley

[rjmackay]

@brandonrosage does this need dev attention? or just design?

[jshorland]

@brandonrosage I'm moving this to design because i'm still not completely clear the desired behaviour here -- is this a bug, ux issue, or new feature?

[brandonrosage]

@jshorland This is the impetus for the "free-form surveys" issue we've discussed over the past few weeks. I'd like for it to be high-priority, and therefore slotted into the earliest sprint possible. If it looks like that'll be possible, I'll prepare a feature branch right away, demonstrating it.

[nickbadams]

! :)

On Thu, Sep 22, 2016 at 2:41 PM, Brandon Rosage notifications@github.com wrote:

@jshorland https://github.com/jshorland This is the impetus for the "free-form surveys" issue we've discussed over the past few weeks. I'd like for it to be high-priority, and therefore slotted into the earliest sprint possible. If it looks like that'll be possible, I'll prepare a feature branch right away, demonstrating it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ushahidi/platform/issues/1335#issuecomment-249036367, or mute the thread https://github.com/notifications/unsubscribe-auth/AFfl9R1NfpBqY7YRwGF1-n4lxLjHit5xks5qsvX6gaJpZM4Jhv_t .

Nick Adams, Ph.D.Research Fellow – Social Science,

Berkeley Institute for Data Science University of California, Berkeley

[jshorland]

Thanks for that context @brandonrosage. let's get this thing built. I'm going to tag it with Sprint 12 for now.

[rjmackay]

Ok.. for the benefit of the spectators (me!) can @jshorland or @brandonrosage add a few sentence scope to this issue? I'm not sure what 'free form surveys' are compared to status-quo.

[brandonrosage]

@rjmackay: "Free-form surveys" is shorthand for an approach to assembling a survey's fields in any order, with only the fields necessary for that survey. So we don't assume things like a "Title" text field, "Description" textarea, or "Location" location field.

The deployer dictates what each of their survey's fields are, what they're called, and what order they're in.

Furthermore, this creates a challenge to the design system with regard to how we display postcards when many won't have a traditional title, etc. The solution we discussed in Montreal involved a modest interface for the deployer to indicate which fields they would want to display in each survey's postcard (and in what layout).

Assuming this feature is clear for takeoff, I'll create a feature branch to demonstrate these things.

[brandonrosage]

@rjmackay, @jshorland, @caharding, @nickbadams: I've completed a wave of initial work and thought on this "free-form surveys" solution. There's a lot of detail I won't cover just yet (like how things change depending on your permissions, or what things default to). I'll try to keep this first brain dump concise.

Configuring a survey

When creating a new survey, here's what you get:

Selecting "Add field" first displays a mainsheet asking you to indicate what type of field:

Selecting "Appearance" displays a modal window for configuring, A, your survey's color, and B, which of your fields you want to display in the postcards (called "response summaries" in the UI):

Selecting "Permissions" displays a modal window for configuring moderation and access:

A complete survey configuration might look like this:

Surveys

So what would it look like to "fill out" a survey?

...scrolling...

Postcards

I actually started this process by dealing with the implications for postcards, since they're one of the most uniform patterns in Ushahidi (e.g. title, description, metadata) -- and making surveys "free-form" would require that they be much more fluid.

Again, the fields displayed in postcards can be one, three, or all of those that are used in the complete form. The UX encourages only displaying two or three, though. The fields that are displayed in postcards can also be positioned in any order the deployer wants -- not necessarily the same order they appear in the complete form.

Finally, the first field (if it's text) is always displayed in a larger font size. If it includes multiple paragraphs, only the first one is treated in that larger size.

So here are some examples of some postcards. Hopefully, the things that remain consistent and fixed are obvious.

Postcard for a survey with a short-text field in the first position, long-text in the second, and an image in the third...

Postcard for a survey with long text as the only visible field...

Postcard for a survey with an image in the first position and long text in the second...

Postcard for a survey showing four fields...

---

Thanks for your feedback and questions, y'all.

[jshorland]

@brandonrosage something charlie brought up that i think we deliberately eliminated in the transition to mustang is help text. We don't have a way for deployers to provide any context or instructional text alongside their custom fields. Ex: In my deployment, ushaverse.ushahidi.io, I added instructions to a survey field as the default text. A contributor has to manually delete that in order to fill it in, making it tedious to fill in fields. Do you think there is a place/compelling argument to re-integrate this back in?

[caharding]

One thing I've found to be a problem about this solution is the continued use of the up/down arrow to reposition questions. It can be very slow and difficult to add a field at the bottom you want to appear on top. Could there also be a click/drag solution?

Jess plus 1's this.

[caharding]

I'm actually also confused about why the Desc/Title field are still treated in a unique way upon survey creation. Are they still a standard entity or not?

[brandonrosage]

@caharding The fields that appear atop a "postcard" are not necessarily title or description. They can be any of the fields in your survey -- the deployer determined which ones in the "Edit appearance" window demonstrated above.

As you can see in the later postcard samples, some of cards show a long-form text field in the top spot, others a short text field for a name, and another places a photo at the top.

[jshorland]

@brandonrosage I want to try to get this into the sprint 14 kicking off two weeks from now. Are you able to get this into Tech spec and sizing during this immediate upcoming sprint (sprint 13)? Is that reasonable? I'm also assuming the design changes for meta data would have to be done at least (not necessarily editable functionality).

[brandonrosage]

@jshorland That's reasonable.

[caharding]

What confuses me is that in edit mode they look like the survey publish view and all the other items are these modal things you have to click to edit. I thought we were going to unify the front end and backend experience of survey creation more tightly. Is that for a future sprint?

Charles Harding

Product Director

tel:207-720-0713

On Tue, Oct 11, 2016 at 2:32 PM Brandon Rosage

< mailto:Brandon Rosage notifications@github.com

wrote:

a, pre, code, a:link, body { word-wrap: break-word !important; }

https://github.com/jshorland That's reasonable.

—

You are receiving this because you were mentioned.

Reply to this email directly, https://github.com/ushahidi/platform/issues/1335#issuecomment-253053039 , or https://github.com/notifications/unsubscribe-auth/ABizaRWIIVKgSOGniSsjW_Qb-isIQMzaks5qzABSgaJpZM4Jhv_t .

[jshorland]

@brandonrosage as soon as you're ready, please talk to @willdoran and @zhalloran about getting some tech eyes on this

[brandonrosage]

Alright, gang (@jshorland, @caharding, @rjmackay, @willdoran, @zhalloran). Here's what I'm proposing for free-form surveys, presented as Jane's workflow for some key tasks.

Create new survey

1. Select "Create new survey"

Jane can begin creating a new survey a couple different ways:

A. By selecting the "Create new survey" link appended to the survey filter (e.g. in Map and Timeline modes).

B. By visiting Settings > Surveys and selecting the "+" button.

2. Arrive at the "Create survey" screen

Upon arriving at the "Create survey" screen, Jane is shown essentially a blank slate for a survey.

It includes editable areas for defining the survey's title and description, and is pre-populated with an untitled "Long text" field. On page load, that field is brought to focus, to help Jane understand how to interact with various segments of her survey.

When in focus, a field has a handful of controls:

• An editable area for the field's label (e.g. "Untitled field").
• A control for changing the field type (e.g. Long text, Checkboxes, Photo).
• A toggle switch for determining whether the field is "Required."
• A "Delete" button, which triggers a modal confirmation window.

It also has a couple controls tucked into a "---" dropdown:

• A checkbox to toggle whether to display an editable field description.
• A checkbox to toggle whether to display the field in response summaries (e.g. known internally as "postcards").

Regarding the "Include in response preview" checkbox, this should be checked by default for the first, pre-loaded field. But it shouldn't be checked by default for any subsequent user-added fields.

Depending on the field type, additional controls may appear, as well. For example, a "Location" field type includes controls unique to that kind of data:

As another example, a "Dropdown" field includes controls for managing the various options:

Note that each option in that case can be re-ordered by dragging and dropping the line item using the "dragger" trigger at its left-most edge.

3. Configure multiple fields

When Jane takes a field out of focus -- either by selecting a different field or simply by clicking outside the currently-active field -- its visual state transforms from showing configuration controls to displaying an approximation of what that field will look like to survey respondents:

She can add new fields to the survey by selecting the "Add field" button, positioned after the survey's last field. Doing so will append an untitled "Short text" field to the survey and bring it into focus.

And she can re-order fields by long-selecting their "dragger" trigger at the top-center of the segment and dragging it into the desired position.

4. Saving her work

There is no "Save" button for the overall screen. Instead, we're employing auto-save, which communicates to Jane through a status-indicator fragment inside the screen's toolbar. Every time a field's value is changed, that change is saved.

When things are saved, the indicator will show:

When things are in the process of being saved:

When a save fails:

It's not clear to me what the scenario for a failed save would be in the case of configuring a survey. But in case there is one, we have a convention for it -- which would presumably couple the status indicator's state with a message module to the tune of:

5. Editing the survey's appearance

By selecting the "Appearance" button in the toolbar, a modal window appears with controls for editing the survey's color.

By default, the survey has no explicit color.

In the context of a modal window, changes to individual fields within it aren't auto-saved. They're saved only after Jane selects "Done" at the bottom of the window.

6. Editing the survey's permissions

By selecting the "Permissions" button in the toolbar, a modal window appears with controls for editing the survey's permissions.

This window includes three controls:

• A toggle switch for determining whether survey responses should be put "Under review" by default.
• A toggle switch for determining whether survey respondents' names will be omitted from the survey's response summaries and detail views.
• A fieldset for determining who can submit responses to the survey.

Regarding the "Who can submit responses to this survey" control, if "No one" is selected, the survey is effectively no longer accepting submissions. The "Preview" button (discussed below) would no longer appear in the toolbar for this survey.

The aforementioned conditions for saving changes within a modal window apply here, too.

7. Previewing her survey

By selecting the "Preview" button in the toolbar, her survey is opened in a new browser tab -- fully functional and ready to accept a submission. If, for example, she completed the survey and selected "Submit," it'd save her response just like any other.

8. Returning to Map mode (or wherever she came from)

Because things are already saved for Jane (assuming system success), she can simply navigate away (via the mode bar) or use the "back" arrow next to the mode context's page title (e.g. "Create survey").

Selecting the "back" arrow button will navigate Jane to the screen that preceded this one -- the screen she was at when she clicked "Create new survey."

Edit existing survey

Jane can edit a survey in her deployment a couple ways:

A. By selecting the survey's dropdown actions from the survey filter in the mode context, and choosing "Edit survey."

B. By visiting Settings > Surveys and selecting the survey's name.

2. Arrive at the "Edit survey" screen

Upon arriving at the "Edit survey" screen, Jane is provided essentially the same experience as "Create Survey," just with a fully-populated and configured survey.

Again, the first field is brought into focus as a hint as to how to interact with each segment.

From here out, the workflow is identical to that of creating a new survey.

How this affects the rest of the client

1. Postcards

Because surveys no longer assume any structure (e.g. title, description), the summaries of their responses (which the design system labels "postcards") can't either.

That means postcards now listen to their survey's structure and configuration for direction on what to include in them.

Here are some examples of variations:

'Long text' field displayed first, along with a photo field

'Long text' field displayed first, along with a 'Multiple choice' field

'Photo' field displayed first, along with a 'Long text' field

'Long text' field displayed first, along with 'Checkboxes' and 'Photo' fields in a Map mode postcard popup

When displayed as a popup in Map mode, postcards have a max-height and are scrollable (as demonstrated above) to access overflow content.

There are some additional-but-important notes to consider for all postcards, too:

• They only display data from fields that were configured to "Include in response preview" (discussed above).
• Those fields are displayed in the postcard using the same ordering directed by the overall survey.
• The first field that's included in the summary should be wrapped in a container with the class name postcard-title and an anchor linking to the response's detail view, giving it a slightly more prominent (if it's text-based) and interactive display.
• "Long text" field values with more than 160 characters should be truncated, with a trailing "...".
• The number of fields not included in the postcards should be displayed last (e.g. 3 more fields...") as a link to the response's detail view.

The front-end pattern for postcards does much of the remaining work needed to dictate when, say, a field's label is displayed, or how things are visually truncated in various contexts.

2. Edit 'Response', 'Role', 'User'

At best, these sorts of entities' edit screens would adopt auto-saving and everything that comes with it, including the "back" button.

3. Metadata

The solution (and its feature branch) leverages the improved "metadata" pattern, which impacts the post status, author and date -- in both 'detail' and 'submit/edit' views.

'metadata', as seen when submitting a response

'metadata', as seen when viewing a response

4. Mainsheet

The "mainsheet" pattern has been available to the client for some time, just not used in any implementation. It's a new convention for handling more complex dropdown-like interactions, like selecting which survey to submit to, or which field type you want to add to a survey.

Submit a survey response

The "free-form surveys" solution leverages (but doesn't require) mainsheets for field type selection. And it's enhanced greatly by implementing it for interaction with the "+" button to submit a new survey response.

---

All of the screens and flows outlined above can be viewed and sampled in the Pattern Library's f-freeformsurveys branch; largely with the following layouts:

• Create Survey
• Edit Survey
• Submit survey response

So...

A, I want your feedback on this solution.

B, assuming this design solution is reasonably solid in your opinion, I think we should do two things:

• Move it into "Tech Spec & Sizing" and get it moving in the client.
• Get feedback from users in a more direct forum, like a Google Hangout with current deployers picked by @aoduor and/or @jshorland. I think there's too much interaction and system feedback going on here to adequately prototype an experience for unguided, anonymous study participants on UserTesting.com.

Let's hear from you: Where do we go from here? What do you need from me to move this forward?

[rjmackay]

I commented on #1469 about this.. but theres a big complication with this when it comes to dealing with data sources. We currently use a posts title and description to save info from SMS, Twitter and Email. Once we get rid of those fixed fields we have a couple of problem

1. Where does the data for unknown posts go? I think this is manageable since we can create a new home for this data in the backend somewhere
2. How do we map data from an SMS/Email/Twitter onto a survey's field? Currently we just automatically copy it into title + description. Now we need to provide users a UI for choosing which fields map to which..

[brandonrosage]

Great question, @rjmackay.

Where does the data for unknown posts go?

I propose that each data source be a pseudo-survey -- it has its own form fields and responses.

The "survey filter" in mode context is a good example of the kind of impact this would have on the web client, for example:

No more "Unknown" posts. Each data source feeds into its own bucket, with its own structure. Though they'd share a lot of DNA with surveys, they'd of course be separated for the user.

Note that, while there's design thinking about how to automate the importing of posts from a given data source directly into a survey, I'm not going to detail that here.

How do we map data from an SMS/Email/Twitter onto a survey's field?

I think any post belonging to a data source would include the action "Convert to survey response." That action would be followed by a UI to "Choose which [data source] field should be assigned to each survey field" -- much like CSV import.

So the mapping would occur (mostly manually) if and when the user decides they want to convert a data source post to a "proper" survey post.

---

Make sense? Does that answer your question?

[rjmackay]

@brandonrosage that makes sense. It probably makes sense as a way to represent this data in the backend too (ie. a pseudo-survey with fixed fields). Look forward to seeing UI for mapping fields, since that could be complex.

[caharding]

A couple things:

1. You're going to need to be able to hide/collapse all Surveys or Data Sources for screen real estate
2. I think that what goes in the "Preview" card should be chosen not on the field, but in the preview mode, where the user can then drop down select which field they want in what order. It is confusing that they would have to reorder the survey if they want to reorder the preview in the current solution.
3. This image's buttons take up too much space, they should simply read "Respond" "Assign" "Show" "Hide" "Edit"

[justinscherer]

@caharding @jrtricafort this should get sorted in the UI refresh.

[Erioldoesdesign]

Wow Okay theres a lot of history here! and I also see that this issue got away from it's original purpose of 'Free form surveys' so it should have really had multiple issues raised for each of the talking points... (from my pov)

Many of the visually confusing elements are solved in the UI refresh, but the interactions the user makes are still overly complex. However, we did implement a better way for moving elements around that isn't up/down arrows in the FE! I just can't find the issue...

I also don't think we eliminated things like mandatory title/description.