search

backdrop-ops / backdropcms.org

Issue tracker for the BackdropCMS.org website
https://backdropcms.org
25 stars 21 forks source link

Usability testing for BackdropCMS.org #492

Open stpaultim opened 5 years ago

stpaultim commented 5 years ago

I'd like to do some simple usability testing on BackdropCMS.org.

As the primary place that people will come to find out information about BackdropCMS, I believe that the ability for users to find what they are looking for on BackdropCMS.org will have a big impact on how likely they are to hang around and/or actually download and install Backdrop CMS.

My plan is to attend future Drupal events or non profit tech events and ask for volunteers to sit down and try to accomplish some common tasks on BackdropCMS.org. I'll sit next to them and watch them and take notes. Hoping that if we have a list of TESTS/SCENARIOS others might also be inspired to do the same.

I've created this issue to get feedback on what some important tasks are for users of BackdropCMS.org and some potential scenarios to test. When people come to the site, what are they looking for?

  1. Report an issue/bug 1.1. SCENARIO: You found a bug with the views modules and you would like to report it. 1.2 SCENARIO: You found a bug with the views_bootstrap module and would like to report it.
  2. Find a simple way to evaluate Backdrop CMS 2.1. SCENARIO: You would like to look at a working copy of Backdrop to see what features it comes with 'out of the box."
  3. Find out how Backdrop is different from Drupal 3.1. SCENARIO: You have worked with Drupal before and have heard that Backdrop might be a better scenario. You are looking for information about how Backdrop is different and whether or not it's the right tool for you.
  4. Ask a question of the Backdrop community 4.1. SCENARIO: You have a general question about how configuration management works and need some help. You would like to post a question to the Backdrop community. How would you do that?
  5. Find out how they can help contribute to Backdrop 5.1. SCENARIO: You have heard that the Backdrop community hosts weekly meetings and you would like to participate, but you don't know when they are and how to participate. 5.2. SCENARIO: You are super busy and don't have time to write code or contribute but would like to contribute $25. 5.3. SCENARIO: You found an error on the following page and would like to report it or fix it yourself. What do you do? (https://backdropcms.org/user-guide/users-roles-and-permissions) 5.4. SCENARIO: You read a tweet asking for help reviewing a specific Pull Request, but you have not done this before and are looking for instructions to help you get going.
  6. Find out whether Backdrop is a good fit for your project 6.1. SCENARIO: You are looking for case studies or examples of Backdrop projects.
  7. Security Concerns 7.1 SCENARIO: You have heard a lot about Backdrop CMS, but are concerned about its maintainability and security in specific. You would like to find out if you will get at least the same level of support as with Drupal. Have past security issues been dealt with in a timely manner? 7.2 SCENARIO: You found a security issue in Backdrop and want to report it privately.

Please, feel free to add questions/scenarios or provide feedback to questions/scenarios on the list. Are these good questions/tasks to be looking at? Do you have other ideas of important tasks that people are trying to accomplish on BackdropCMS.org that we could test?

stpaultim commented 5 years ago

As I look at these tests it occurs to me that we could simply invite friends to go through a list of scenarios on their own and provide us some feedback on how easy it is for them to accomplish these tasks?

quinnanya commented 5 years ago

Another task idea: You want to see what sites have been built with Backdrop (to see what they look like, and what kinds of sites they are), to decide whether it's a good fit for your project. (Relatedly: you'd like to see how a site that seems similar to yours was built -- using what modules, what approach to theming, etc. But this might be a big ask for anyone to do on the documentation side.)

Looking at the site more carefully, it looks like there's been a lot of effort to make this prominent. But to be honest, if I ever scrolled down on the front page before, it didn't stick that the second block of content included examples. And I'd filed away the word "showcase" as probably referring to technical features rather than sites.

stpaultim commented 5 years ago

In my limited experience with usability testing, I find that it often reveals how things that appear obvious to us in hindsight are not nearly so obvious to first time visitors or beginners. I have experienced this myself, trying to find things on the Backdrop site and failing only to come back later and find it fairly prominent.

klonos commented 5 years ago

Scenario: I have heard a lot about Backdrop CMS, but I am concerned about its maintainability and security in specific. I would like to find out if I will get at least the same level of support as with Drupal. Have past security issues been dealt with in a timely manner?

stpaultim commented 4 years ago

In conjunction with our Infrastructure Sprint this week, I have created 4 unpublished "self directed usability tests" in the forum using 4 of the scenarios mentioned in this topic. The idea is that we use Twitter to ask for folks willing to try any or all of the 4 scenarios and report back to us on their experience.

I thought we could try this in the forum and see how it goes. We could either leave these issues open after we are done or close comments on these specific issues and leave them there archived. We could ALSO unpublish them, but I don't think that would be necessary.

Any thoughts on the value of this kind of testing and/or whether or not this is good time to conduct them. I recognize that only limited people have access to view unpublished nodes on forum.backdropcms.org, so I will include the full text of one of the tests.

The 4 tests I have proposed are (open to discussion - we could do different 4 tests or we could do more than 4).

  1. Usability Test - Report a bug for contrib module
  2. Usability Test - Evaluate working version of Backdrop CMS
  3. Usability Test - Find information to help compare Backdrop CMS to Drupal
  4. Usability Test - Finding security information

NOTE: I have left these nodes unpublished for now, until I get some feedback on this idea.

FULL TEXT OF TEST - 2 (example):

We are asking folks to help us conduct a usability test on our community infrastructure as part of our infrastructure sprint:

How to help:

If you have not had the opportunity to evaluable a working copy of Backdrop CMS in the last 12 months:

  1. DO NOT read the comments here until you finished the test. 
  2. Attempt to accomplish the following scenario
  3. Post a comment below with feedback about how easy or difficult it was to accomplish this task and any ideas you have for making it easier.

Scenario:

You would like to look at a working copy of Backdrop to see what features it comes with 'out of the box" without having to install it yourself. 

klonos commented 4 years ago

Thanks for this @stpaultim ...yes, I find it valuable to get feedback on these sort of tasks 👍

jackaponte commented 4 years ago

Yup, thank you for this @stpaultim -- this seems like a good idea and a good approach to me!

olafgrabienski commented 4 years ago

You would like to look at a working copy of Backdrop to see what features it comes with 'out of the box" without having to install it yourself.

Just a quick feedback about the language: As not native English speaker, the term "working copy" is confusing me a bit. Is the scenario about a "demo site"?

stpaultim commented 4 years ago

@olafgrabienski - How about "sandbox version"? I edited the copy above to show this.

I don't like demo version, because it is to passive. It is not clear to me that I can login and make changes to a demo site.

olafgrabienski commented 4 years ago

@stpaultim Yes, "sandbox version" sounds better to me. (I agree that "demo" might be too passive.)

stpaultim commented 4 years ago

We talked about this briefly after the meetings today and went ahead and published the first 4 tests. @jenlampton suggested that we create an entire category for usability tests, which I think works well.

https://forum.backdropcms.org/forum/usability-studies

I will not publicize these for a day or two, so it's not too late to offer feedback on the wording - or to suggest alternatives. I recommend having no more than 4 active tests right now, but we can add more later once we see how it goes.