Final report - BETA

[lauradereynal]

Everything that could not get done in Alpha, little details remaining, media etc.

280

General

• [x] Make all the titles the same size
• [x] Update the content structure, and the TOC
• [x] Add a TOC spread
• [x] Make each title a memorable sentence (instead of just a phrase)
• [x] Add the graphs from the comprehensive analysis
• [x] Point to the dashboards more frequently to show people that we have way more data
• [x] Add Key Takeaways for each sub-section to make sure they have a point
• [x] See if each section should have a conclusion spread (like the adoption one does)

Media

• [x] Clean up all the graphs so that they're the right colour and don't have drop-down menus in them (they're too obviously screenshots right now)
• [x] Fix image links (put images in the images folder)
• [x] Make the map with all the data
• [x] Decide about infographics data ?

Section 1 = exec, intro and methodo

• [x] Needs a short description when you open the report: what are you about to read? (reorganize?)
• [x] Make a new executive summary according to Sam's comments.
• [x] Is there a scoring mechanism for Internet Health? How does Kenya's compare to other countries ? How does it affect individuals?
• [x] Move internet health to environment (chapter 2)
• [x] Fix C&J's logo (if they really want...)
• [x] Finish the methodology section and include limits (android data collection and contamination of the control and treatments )
• [x] Connect internet health section to broader mozilla program

Section 2 = who are first time smartphone users

• [x] Add a lot of data from interview one in the form of graphics and not infographics.
• [x] Add the vulnerability and need to belong in the participants description Add paragraph about how people don't want to feel left out, and how they're left out in society; they don't want to feel left out in the digital world; people are vulnerable, have to fight everyday; they want to feel included, not left behind. they're already disconnected; product designers should be careful not to exacerbate the problem with the growing digital divide.
• [x] Add more data about DFS usage amongst participants

Section 3 = Adoption

• [x] Make "why people bought smartphone" graph bigger and prettier
• [x] If you can use whatsapp on your feature phone, why do people want to use it on a smartphone? experience on a featurephone isn't what people are seeking. no selfies, camera, images, videos, etc. the can use featurephones, but they're going to try to get smartphones anyway because of those features; they don't wan to be left behind; feature phones aren't really a choice—if you can avoid them, you will.
• [ ] Contextualize why some women get into betting; what are their motivations? do they not want to be poor, or want to take part? something else?
• [x] Gambling is a concern for people in this line of work (e.g. foundations). people use mshwari to get loans for betting. it's a big deal; people are looking for consumer protection solutions.
• [x] Lottery-based savings already exist in the US; designing systems where you always win, but there is perceived risk (i.e. same feeling as gambling)
• [x] expand on section about hardware manufacturers and what they can do to improve the smartphones that people have (improve the experience); dave's team is working on this problem: how to design a smartphone for this population? these devices aren't adopted for this audience, so if we were to design a smartphone for them tomorrow, what would it look like? "reimagine what a smartphone could look like"; "reimagine the smartphone" => to do with @secretrobotron
• [x] add graphs about how many people had to get a new smartphone over the course of the study
• [ ] idea: using sheng as a language that can be used as a language that you can add to android?
• [x] technical language is difficult--make the point clearer; imagining a new phone, how do you avoid language that is difficult?
• [x] do women spend more than men in data bundles? how does their usage pattern different in terms of money? do the different behaviours they have translate financially?
• [x] don't mention Sterro in quotes without explaining
• [x] pre-installed applications: if we worked with manufacturers to pre-install DFS applications, there are problems to avoid, but it's not necessarily the case that it won't work. you can do it with the right approach and the right conditions (and training) in place, but don't assume that it will be used just because it's pre-installed.
• [x] people have trust in institutions: if it's there, it's vetted. need to teach people the critical thinking
• [x] opposite story: how they aren't exposed to how open the web is. there are few things like this in the world. intrinsic openness makes them behave strangely
• [ ] Add picture of gambling machine and explanation
• [ ] Add Duncan's graph to the Digital Credit case study, because it's a lot more intimate than the one Jurian built. It has a much shorter time scale, so you can see the dispurse/repay action really clearly.
• [ ] Add quotes from https://docs.google.com/document/d/1Zh8IGpBZTVymllILLAc6PEDalveimseHlTo0ip86ZDU/edit that @LauraReynal pointed out
• [ ] Turn “Using social media for business” into report content

Section 4 = skills and methods to increase adoption

• [x] The Impact section of Chapter 3 feels a bit repetitive from the information that preceded it. We should try to include things like this as well to show that there was overall impact more strongly: https://www.dropbox.com/s/et777t3uhajk39e/Screenshot%202017-01-17%2021.20.37.png?dl=0
• [ ] Add analytics tables from Jisort as a supporting evidence in the findings section https://docs.google.com/spreadsheets/d/1Cq1bn3mfhOEo3FsZ3WtmiRxo7mjmmoj25s3-r7HnOLE/edit
• [x] Jisort: make a "What is Jisort section which condenses design + content and is shorter" Follow closely with findings. Put them in the appendix.
• [x] Sterro: make a "What is Jisort section which condenses design + content and is shorter" Follow closely with findings. Put them in the appendix.
• [x] Workshops: make a "What is Jisort section which condenses design + content and is shorter" Follow closely with findings. Put them in the appendix.
• [x] Add better hourly, dately, and personal graphs from Sterro: https://docs.google.com/spreadsheets/d/1T6cEotkE9EoMrZRHw-WFrRDZSVMh34_2PL-Emk78JF8/edit#gid=1086134124 @secretrobotron
• [ ] Make sure "most used URLs" table is properly reflected: @secretrobotron https://docs.google.com/spreadsheets/d/1YsLR6S9ICQcNOSlB1V9TJOW_pu944_0BCgbyWeazz6I/edit?userstoinvite=laurad@mozillafoundation.org&ts=587a3739#gid=0

Section 5 = impact

• [ ] Move the paragraph about the evaluation of digital skills into the section Digital Skills lead to Confidence, Awareness, & Ability
• [x] Add All the data from the evaluation, where I highlighted in yellow @secretrobotron
• [x] Review other paragraph highligjted in yellow
• [x] Finalize the titles for impact: impact of methods, impact of skills, impact on DFS ? There must be another way. Its missing the words: "Product design" but maybe that goes in the conclusion
• [ ] Add the great work from https://docs.google.com/document/d/1FlZ3x7rwtscTw3hK-TRaaWXDQfRsvD9HcMj12ZW3x2U/edit# after the 3 impact spreads. Title the new section: implications, recommendations and opportunties? ?? ❓ @secretrobotron the wording is pretty important on this section
• [ ] Add quote from community member about the program's impact on them. Pick from the interviews.
• [ ] make sure all the content from the spread called "review of the research question" is in the implication section
• [ ] Circle back to internet health ?
• [ ] add our website and dashboards again at the end ?

[secretrobotron]

👍 let's write down review feedback here too

[lauradereynal]

It became obvious to me that the spread "Evaluating digital skills" must be back as a normal text. But let's wait for the review and do this for the beta.

[secretrobotron]

@LauraReynal added all the feedback we thought about so far to this bug instead of #280.

[lauradereynal]

Something I thought of but didn't get to check for the recommendations section: you wrote a lot of good recommendations for the mid study report that had call outs for NGOs and governments to help. Can we revive that? I think it would be useful to get people thinking at that level after reading our report, especially if they're attending our convening.

Over and out.

[secretrobotron]

• [ ] Perforated design challenge cards as "take home" activity?

[JoashMango]

I dont want to repeat what my colleagues have mention. My review is

• The quality of the graphs need to be improve. This should be the same as in the silk

• The format of the chapters. Take a ways should be in the start of the chapter. This will give the reader the urge to know more by reading the chapter

• We had talked about digital skills people need on page 44. So on Evaluating digital skills on page 74 we need to add how we evaluated the digital skills that were required in page 44

[lauradereynal]

@JoashMango thanks for the review and the feedback. As we mentioned in the email, the design and the presentations are not done at all, so the titles, the chapters and the graphics are going to be improved soon.

Regarding the skills, thanks for catching that. It's indeed very important. We are going to expand the evaluating digital skills section so that it's more representative of the whole evaluation.

@JoashMango Do you have more feedback about the content of the report? The different theories and findings? Did everything make sense for you ?

[lauradereynal]

@secretrobotron

From #44: write about information sharing between control & treatment group

=> yes that's a good one, but where? We don't have a risk section, and it's also a finding... Maybe in the section about how people teach their peers more ?

From #250, add a note at the end of the report saying that this was an "open" as we could make it, and that you can find all of our materials, source, interviews, etc. online.

Yes, this will be especially good for the appendix

[secretrobotron]

Maybe in the section about how people teach their peers more

I really like that as a finding. Good thinking. We should also add something small in the methodology section though.

[secretrobotron]

• [x] Along with a more detailed map of participant locations, we should add a story about how we changed our tactic slightly to get more people to show up. IIRC, we did workshops in Kisumu sub-regions, right?

[secretrobotron]

The Cyber Cafe case study is great. I think we should be sure to mention them in the "Environment" section of Chapter 2, since they're a place that presents adoption, barriers, and the potential for skills growth. We can publish the full case study on the website and refer to it.

What do you think, @LauraReynal ?

[secretrobotron]

From @DuncanGW's review

Chapter 1

• [x] (pg 5) In Internet Health, Digital Inclusion: soften language for women. It says "access to these aspects society is nevertheless impacted by the decisions of the men in their lives.", but 60% of respondents are single.
• [ ] (pg 10) In Methods & Approach, Interventions and RCT: "but also the attended" should be "but also attended" (get rid of "the")
• [x] (pg 12) In Methods & Approach: Move the "Interventions" section to Teaching Methods section (@secretrobotron thinks we need to discuss Interventions as methodology here, but perhaps in less detail. We can describe them briefly, perhaps.)

Chapter 2

• [ ] (pg 75) In "A closer look on digital credits": "which becomes more affordable and easier for him" should be changed to "worth it" (@secretrobotron: maybe another phrase) because he is trapped in a debt cycle and Tala loan is expensive compared to M-shwari
• [ ] (pg 75) In "A closer look on digital cretits": "happy to increase his credit score." --> We discovered its not to increase his credit score but to settle his other debts and contribute to savings group.

Chapter 3

• [ ] (pg 119) In Sterro: Update "Messages vs Date" graph, because it's not clear why it's there.
• [ ] (pg 129) In Sterro: Update "Response Rates of Various Participants" graph, because it's not clear why it's there. (@secretrobotron: this needs its own small discussion. maybe a new spread to talk about the different trends)
• [ ] (pg 132) In "Impact of Methods": the "Please rate which taught you the most" is unclear (@secretrobotron: It needs a caption, perhaps. I don't know if it's clear why it's included in that section.)
• [ ] (pg 140) In "Digital Skills lead to Confidence, Awareness, & Ability": The Treatment Confidence graph has a different order than the Control one. (@secretrobotron: there's not a lot we can do about it because that's the way silk generates graphs, but I can just doctor it in Photoshop if it's really confusing.)
• [ ] (pg 158) "Research Questions Review": The title has a spelling mistake (Research Questions R§eview)

[secretrobotron]

• [x] Change name for the HIV story

[lauradereynal]

From Dave

• [x] Add something more meta about the open nature of the internet and the trust in institutions => Many first-time smartphone users like Esther are not fully aware of the open nature of the internet and do not assume that content can be created by anybody with access. (here do not assume we can say do know)

• [x] Tian Page 19 (the sentence needs to be rephrased) Sometimes because of his position in the community, he is entrusted people's phones to will help them write or respond to messages they are too busy or otherwise ill-equipped to manage on their own.

[secretrobotron]

List of words that need to be consistent:

• participants (vs. respondents)
• preinstall (vs. pre-install)
• internet (vs. Internet)
• web (vs. Web)
• Digital Financial Services (vs. digital financial services)
• Conjunctions are bad (e.g. won't --> will not, can't --> can not)
• gambling (vs. betting)
• WhatsApp experiment vs Chatbot experiment

[lauradereynal]

Cool quote:
He would like to learn about instagram application, how it operates and how he can use it to connect with friends. He would also like to learn how to create his email and other people's email addreses

[lauradereynal]

• [x] http://dso-interview-1.silk.co/page/Technology The adoption graph has a couple of mistakes in the code, where there are a few "social networks" hanging at the end !

[secretrobotron]

• [x] "Where do you get your betting cash" isn't coded properly in http://dso-interview-5.silk.co/page/Gambling-%26-Betting

[lauradereynal]

@secretrobotron We need to edit and arrange the style of the smartphones ownerships graphics: Android versions and hardware.

1. Can you put them in Silk so that the style is similar ?
2. Additionally, the phones should be edited for the following mistakes:
   • M3 is tecno phones, so merge with tecno
   • Huawei is present twice with different colors

[jurb]

"Where do you get your betting cash" isn't coded properly in http://dso-interview-5.silk.co/page/Gambling-%26-Betting

@secretrobotron This is now fixed.

http://dso-interview-1.silk.co/page/Technology The adoption graph has a couple of mistakes in the code, where there are a few "social networks" hanging at the end !

@LauraReynal Can you check the link again? I can't find a adoption graph on that page, or on the dashboard 1 anywhere, so I think you mean a different interview.

[lauradereynal]

• [ ] Interview 5 > This drop in attendance is attributable to the female respondents: only 60% of the women attended the fifth intervention workshop.
• [ ] Interview 5 Jisort: 73% of the respondents have learned something from using the Jisort app.

[secretrobotron]

• [ ] Sheng is used but not defined on page 5
• [ ] orphan on column 2 on page 6
• [ ] colors on right side table of page 6 aren’t up to the rest =)
• [ ] page 7: “and teachin for g them how”
• [ ] “• How do I downloading, use, and update apps?”
• [ ] p 12: recieve -> receive
• [x] p. 16 — I don’t think Esther has a salary — it’s daily income, not salary.
• [x] p. 17 - “People finds ways to share” — people find ways
• [ ] p. 19 — “and is widely regarded”
• [ ] broken bullets on p. 21 and 22
• [ ] p. 22 - “benchmark that is often used in our work.”
• [ ] p. 23 — I don’t understand what kind of betting is popular — is it races, online games, etc? Helps explain the need and value of doing research with smartphones.
• [ ] I would remove the control vs treatment group graph, given there’s no difference.
• [ ] p. 23 — “Over time, there were differences between our treatment and control groups as well, but no sustained difference.” — rewrite
• [ ] p. 23 — entertainement is mis-bolded
• [ ] p. 24 — “Because it’s got such” —> because it has such
• [ ] p. 27 — “are inevitable, and people have grown accustomed to at the first sign either in an attempt to regain a consistent experience.” ??
• [ ] “Blogs and forums are offer explanations and alternative”
• [ ] “Most people did not know that smartphones there were other operating systems to use”
• [ ] “It was essential THAT DSO’s digital skills workshops”
• [ ] “This imbalance must have an affect” -> effect
• [ ] p. 29 — legends on charts hard to read
• [ ] p. 29: “From DSO, there are several stories of more direct impact on women’s technology use from their male counterparts” — rewrite
• [ ] Did you try single-gender workshops?
• [ ] is Sterro a male name?
• [ ] “At the beginning of DSO, the majority of participants at DSO (93%) “ — I would get rid of “at DSO”
• [ ] [get someone to do a copyediting phase on the whole document]
• [ ] p. 68: “mobile pension” (not mention)
• [ ] p. 69 what is the chart about?
• [ ] p. 70 — digial -> digital
• [ ] p. 80 - header is messed up
• [ ] p. “150?” - Mozilla volunteers teachers — some strikeouts there?
• [ ] p. 158 — heading
• [ ] p. 11 - Pursuie
• [x] p. 162 : “Finally, we are forever grateful to the 200 individuals who shared their lives—their joy and their struggle—with us as we. “ missing some last words?

@LauraReynal

[secretrobotron]

• [x] Make sure we use "income" instead of "salary" throughout. ==> I did a search and replaced.

[secretrobotron]

• [ ] Check on Canadian/UK/USA language (e.g. neighbour vs neighbor)
• [ ] Check on "low-income, first-time". Sometimes we use hyphens, sometimes not.

[secretrobotron]

Some feedback on section 5 for @LauraReynal:

• Overall, I made some language corrections, but it needs a thorough pass (e.g. Kev)

• "In person workshops are the most preferred and impactful" has a little bit of information about impact, but it's pretty focused on comfort. Can we add some reasoning about why participants enjoyed "group discussions, friendly staff, face-to-face interaction, the variety of learning activities, and the open Q&A troubleshooting sessions" the most?

• About the title, "The WhatsApp experiment was the second most impactful method": this is true, but maybe a more useful title is something like, "The WhatsApp experiment was more useful for a smaller audience, and could be a viable alternative to workshops."

• Maybe, "Stand-alone native mobile applications do not seem to be the right path for this audience." could be "Stand-alone native applications are less feasible".

• I see that there is a lot of repetition here now. For example, recommending how to use chatbots, and giving better, more local, community-run workshops. They're both mentioned in the first part of section 5, and in the conclusion I started writing. Let's just pick a place where they fit best. Since this first section is really a comparison, it probably makes sense to move all of this content into the conclusion/recommendations section.

[secretrobotron]

@LauraReynal I found this floating in the ether of a textbox:

Key Takeaways

A lack of confidence can hinder people, make them too shy to speak out and ask questions. Designing a learning program specifically for them, at their own pace, had a positive effect on people’s agency and sense of empowerement.

This means that they are more likely to explore by themselves, without being too scared of getting into trouble or breaking their phones.

=> How might product designers model these results to include features in their product which will leave customers with a sense of empowerement?

( How does this become embeded into product design? What does confidence looks like as a feature?)

=> ?

Design and deliver the training in the local dialect, with as little written content or activity as possible. Provide food, a free Wi-Fi connection, and electricity. If possible, have a space designated for babies, or even a baby sitter, to make sure mothers are able to focus on their learning. Make time to help people troubleshoot problems they face on their devices. A workshop might turn into a "helpdesk" before you realize it!

[lauradereynal]

@secretrobotron answer to your comments from yesterday

"In person workshops are the most preferred and impactful" has a little bit of information about impact, but it's pretty focused on comfort. Can we add some reasoning about why participants enjoyed "group discussions, friendly staff, face-to-face interaction, the variety of learning activities, and the open Q&A troubleshooting sessions" the most?

Very good catch. 👍 I have changed the whole text completely, and added points about the impact of using participatory practises, the impact of conversations, the impact of peer to peer learning ... Can you check out that whole section again please? I think it needs your eyes one more time.

[lauradereynal]

hahaahhaaah non sense. This is definitely not closed.

[lauradereynal]

Additional:

• [ ] re-work the conclusion of the excecutive summary to follow more closely, our conclusion to date, which is much stronger.

[secretrobotron]

• [ ] On Page 27 the Orange pie chart key states that, "Just 3 Digital Financial Services providers make up 75% of the market share among participant." MPESA, Okoa Jahazi and M-Shwari are all products of Safaricom LTD. Would it be important to add that so that readers know that?
• [ ] Page 18 states "...broke screens, tactile keyboards emojis and Safaricom." I find Safaricom which is an MSP is out of place. The other items inherent characteristics of the devices new users use to get to the web but Safaricom is not.

[secretrobotron]

Some great notes from @Uwanja. Let's make sure we have these covered:

• [x] "..and forums are offer explanations and alternative, less disruptive methods." I think word "are" shouldn't be there
• [x] On the same page this sentence doesn't sound right. "...Most people did not know that smartphones there were other operating systems to use, and thought that smartphones all ran with the same technology."
• [x] On page 57 "..As they come online, women’s smartphone is continually affected the men in their lives.."
• [x] "..This imbalance must have an affect on the mindset of.."
• [x] On page 71. "..message me about registering me in the RB." Should be CRB at the end which stands for Credit Reference Beureau.

[lauradereynal]

• P5: The blue line does not touch the edge of the page – Footnotes smaller – The footnotes changed the alignment of the white boxes of the executive summary – p14: the font is of different sizes: 9.5 and then 9. Is that normal ?
• P.20 @secretrobotron can you check the URL I added for Jisort ? Is it the right one ? For Mark: we need to make it underlined and blue ?
• P.20, need to add the real page numbers –P.26: See more in P.X environment: need to be adjusted with final page number

[secretrobotron]

P.20 @secretrobotron can you check the URL I added for Jisort ?

I think that's a good enough place for now. I can change the mzl.la URL later, but the github page has the latest release for download.

[secretrobotron]

"The ecosystem of product design for first-time smartphone users often refers to the global trend of rapid smartphone uptake." feels a bit too wordy for me. You, @LauraReynal ?

[lauradereynal]

@LauraReynal Not sure what footnote was supposed to be referenced after "Gender affects technology use in many countries, including Kenya", but I used  http://www.pewglobal.org/2016/02/22/internet-access-growing-worldwide-but-remains-higher-in-advanced-economies/

YES

[lauradereynal]

@LauraReynal there is a missing footnote on page 58 after "Within the societal norms that can make traditional learning methods more difficult for women". Can't find it in the original doc either. You had a article link?

FIXED - no article

[lauradereynal]

"The ecosystem of product design for first-time smartphone users often refers to the global trend of rapid smartphone uptake." feels a bit too wordy for me. You, @LauraReynal ?

Good to me

[secretrobotron]

🎉 🎈 🎂