Get Community Input on Visualisations

[notconfusing]

After creating a UI mockup promote and fish for input on if the site and visualizations look right.

From @fsoong: I would like to do separate initial survey to understand the editors' experience using current analytic tools such as http://reportcard.wmflabs.org/, http://datavis.wmflabs.org/, http://www.wikipediatrends.com/, ones outside Wikipedia such as Google Analytics. I also want to know how they prioritize the graph and chart options we currently have. And what kind of time frames / snap shot points for the data would be most useful.

[fsoong]

Because I am starting a part-time job this week and will be quite busy too this month, I'm very open to other people helping us design most of the actual components (in Photoshop, Illustrator, HTML/CSS, etc.) and I can focus on:

• Coordinating collaborative design efforts
• Gathering community input
• Establishing cohesive art direction

Let me know if you're interested (directly hangouts message/email me fs.soong@gmail.com), we can break down one or a few subissues for you together, and I can assign it to you.

[QEDK]

@fsoong Can I be the one gathering community input? I'm quite active on enwiki.

[fsoong]

@QEDK I don't see why not

[Masssly]

Sure, will discuss that in detail with my team.

Sent from Windows Mail

From: fsoong Sent: ‎Sunday‎, ‎May‎ ‎10‎, ‎2015 ‎8‎:‎25‎ ‎PM To: notconfusing/WIGI

@QEDK I don't see why not

— Reply to this email directly or view it on GitHub.

[Masssly]

The target audience to receive input from (to publicize any surveys or mocks) I can think of right now would most likely be in the research, analytic and developers mailing lists. @QEDK will provide us with a template of what to send so we can all review it.

[fsoong]

Here is a typeform survey I've emailed a beta test group to gather feedback on mockups: https://experiencedesign.typeform.com/to/LNKOuP. This was after an initial longer survey was sent to the whole community with more demographic details (similar to the one I have on Trello) to establish that beta test group.

[Masssly]

I think you should go ahead and use it, the feel looks cool to me. Please refer to the suggestions I made earlier below:

• Because a high number of people will not complete the questionnaire (reasons include lack of interest, lack of an incentive, time constraint, fatigue, etc.), consider prioritizing which questions come first so even if a prospective respondent drops off whiles answering, there’d have answered the most pertinent question (typeform forms captures incomplete surveys right? Others do). I’d suggest this order: B,C&A (explained below).

• “This initial survey should only take about 5-10 minutes to answer. It is a 3 page survey with no more than 5 questions on each page for a total of 13 questions.”

The first page is cool. Give an extra one/two line summary of what WIGI is. Delete the word “initial”, it gives the impression that more surveys are on the way and that alone is enough to scare some people away to begin with. If it is possible, compress the survey to 5 minutes: that would mean doing away with the least important questions and minimizing the use of open-ended questions as much as possible or at least not make them compulsory. People do not like answering surveys; 5-minutes tick/click surveys usually have high completion rates.

I would rather go with “This survey should only take about 5 minutes to answer. It is a 3-page survey comprised of [exact number] questions.

Moving further from my recommendations, now lets categorize the survey into three pages like you've done already; A,B&C.

A. Knowing the respondent Motivation: We can assume that those who are more experienced about Wikipedia will provide the most useful answers. Below are the questions that pertain to A;

A1. How often do you edit Wikipedia? (Closed: 0-1x a month, 2x a month, 2-5x a month)

A2. How long have you been editing on Wikipedia? (Closed with no. of months: 0-3, 4-6, 7-12, 1Yr to 2Yrs, more than 2 yrs)

A3. Would you like to hear more about the WIGI project?

A4. Can we email, chat, or call you to follow up on your survey response?

B. Knowledge of other analytic websites Motivation: Aside experience in Wikipedia, respondents who have encountered similar analytical websites will have a better understanding of what WIGI is trying to accomplish. They are in a better position to give advice based on their experiences. Below are the questions that pertain to B;

B1. Which Wikipedia analytic website (tools?) have you used? (Tick as many)

B2. How have you used the tools? (Tick as many)

B3. Which general analytic tools have you used? (Tick as many)

C. What they would like to see Motivation: Now is the time to actually ask the most important question of what they would like to see on an analytical website, in this case WIGI. What did they wish was available from visiting similar sites in the past?

C1. What do you think the Wikipedia Gender Inequality Index (WIGI) is analyzing? “I am not sure I understand the relevance of this question, please explain”

C2. How much time would you like to spend understanding gender inequality in Wikipedia articles? (Closed, please include 2 minutes)

C3. How much time would you like to spend writing or editing Wikipedia articles about women? “I am not sure I understand the relevance of this question, please explain”

C4. Please check off the graph options that you would be most interested in viewing. (Tick as many)

C5. What insights would you be most interested in gathering from WIGI? (Open)

[fsoong]

@Masssly Are you confusing the surveys up? It looks like you've pasted your questions from Trello and have only changed just "google form forms captures incomplete surveys" to "typeform."

On this question, I don't think so.

For that reason, I don't recommend using Typeform for the initial 3-page survey. However, with Typeform it's easier to provide images right in the questionnaire, and make a quick 5-question screener survey for followed by more in-depth-up telephone or chat interviews. It's been shown (from my personal experience as well) that these screeners are effective in making sure you're speaking to your target audience, but also in making people more interested in doing the interview in the first place. It's more effective than directly emailing people, making a post to ask people, or making an ad to ask people to give their open-ended feedback.

[QEDK]

I would say that we select certain users and e-mail them. Well, I can make the templates and stuff but first you all have to decide the mode of communication. And if it's an e-mail as I suggested and not mass-posting, then there's no need of a template. Best to be informal.

[Masssly]

@fsoong I just re-read your earlier comment. I now see that the two surveys are different. :smile: But you haven't shared with us updates on the initial 3-page survey, I believe there were some changes you had to make? Or is it ready for posting?

@QEDK Reaching users by email at this stage seems intrusive, they must have opted for it somehow - like they are comfortable doing a followup with us in the future and receiving emails. Also the mailing list has the potential of reaching more people. But we are presently working behind schedule so I don't want us to spend more time discussing this. If u think "select[ing] certain users and e-mail[ing] them" too will work that's okay. So produce a list of users (plus the criteria for selection) and lets get their input ASAP. We should begin summarizing survey responses by 21st May.

[Masssly]

@QEDK Are you still working on this task? It was your idea to go informal. Since I cant get to you via email to discuss the survey questions too I 'll take care of this myself tomorrow morning.

[QEDK]

@Masssly I meant if we e-mail people, we should rather be informal. I'm getting templates up ready. I hope you have a test group of sorts.

[fsoong]

Just in case other people (besides QEDK and Masssly) are wondering, I wasn't able to make edits Masssly suggested. Instead, I've given editing permissions of the google survey to Masssly and QEDK. I look forward to reviewing the final template!

[notconfusing]

@Masssly the main point of the survey, is to find out what kind of information users would like captured, ideally about the state of Wikipedia biographies. We will use it to inform what statistics our portal will show.

From the google survey I would suggest these changes:

1. More of an introduction. Introduce WIGI using text from our project page https://meta.wikimedia.org/wiki/Grants:IdeaLab/WIGI:_Wikipedia_Gender_Index
2. Delete Question "When was the last time you edited an article on Wikipedia? What did you edit?" or merge it into user info on page 3.
3. Delete "Which general analytic tools have you used? " and "What do you think the Wikipedia Gender Inequality Index (WIGI) is analyzing? " I don't think they will be useful to us, but...
4. I would add a question like "How well do you think the Gender Inequality on Wikipedia Biographies reflects Gender Inequality in the World?"
5. On page 2 move "Please check off the graph options that you'd be most interested in viewing: " to the first question and change it to, "Which additional variables are you most interested in?" (a) date of birth (b) date of death (c) citizenship (d) ethnicity (e) Article in which Wikipedia languages (f) profession.
6. Add a Question on page 2, "Any other variables that you would like to see included in the the inequality study?" (Free text field).
7. Then on page 2 have "How much time would you like to spend understanding gender inequality in Wikipedia aritcles? " and "How much time would you like to spend writing or editing Wikipedia articles about women? " and "What insights would you be most interested in gathering from WIGI? "
8. Page 3 is great, just combine the editing experience into good multiple choice categories.

[QEDK]

Fixed one question. Fixed typos. Added data validation to some questions. Moved a few questions. Changed fonts. Made a few questions compulsory. Fixed a few options, moved them to end, fixed grammar and stuff.

[Masssly]

@notconfusing @fsoong @QEDK Been offline past couple days, will respond to you all shortly. Thanks!

[Masssly]

@notconfusing @fsoong @QEDK @hargup @vivekiitkgp

Hello guys, I lunched the initial survey here: https://wikimedia.qualtrics.com/SE/?SID=SV_9ZXvma8y7fBfMMZ and sent out some invitations to potential respondents. Please help out by forwarding the link to as many people as people and let me know if you want anything else removed/added/changed.

You might want to copy and paste this text when you are inviting someone to it:

Dear [Name of person]:

WIGI is an Individual Engagement Grant that seeks to automate the production and graphing of statistical presentation of gender in articles by date of birth, place of birth, citizenship, ethnicity, occupation, and Wikipedia language - in a publicly viewable website with open-data downloads.

The purpose of the survey is to find out what kind of information users would like captured, ideally about the state of Wikipedia biographies. We will use it to inform what statistics our portal will show.

We will provide a final report on the observed trends at the end of the year. Please help us out by completing this brief survey, which should take not more than 5 minutes to complete.

Again, thank you for your time and cooperation.

Best Regards, [Your name]

[Masssly]

Oops, sorry here is the right link: https://wikimedia.qualtrics.com/SE/?SID=SV_3OueVOHoRpezvZH You can complete the earlier one too :grinning: its still running.

[fsoong]

Oops, I read this so late. If I had gotten here earlier, I'd have argued for keeping the "when was the last time you ___?" because it's a standard UX / design research starter to gauge the validity of the person self-reporting.

Q1 none option missing Q2 should add "if you have selected any tools above,"... and provide an N/A option. Q3 should remind people the description of the project is on the first page and/or provide a link to our grant proposal. also provide a "not sure" answer.

[fsoong]

We can open another issue for when we do more. Here's public links to Masssly's emailed output for now: https://www.evernote.com/l/ANvaYIYAP9lFrZQysPQQbKkfS_hPWBehJn0

[Masssly]

Hello Frances,

Thanks for your upload of the “Initial community feedback survey” onto Evernote. I am concerned about the privacy of the respondents since it contains their personal contact information. There’s another version that does not, which I’ll send to you soon so you can replace it.

Many Thanks

-Masssly

Sent from Windows Mail

From: fsoong Sent: ‎Friday‎, ‎July‎ ‎10‎, ‎2015 ‎7‎:‎13‎ ‎PM To: notconfusing/WIGI Cc: Mohammed Sadat Abdulai

We can open another issue for when we do more. Here's public links to Masssly's emailed output for now: https://www.evernote.com/l/ANvaYIYAP9lFrZQysPQQbKkfS_hPWBehJn0

— Reply to this email directly or view it on GitHub.