Open marmarta opened 4 years ago
how comfortable you are with using command line tools
how comfortable you are with Linux
There must be a better way to phrase this question. If we phrase it like this it might give users a false impression and scare them away from giving Qubes a try.
My current idea for the questionnaire:
INTRO: We want to make Qubes as user-friendly as possible - and to do that, we would like to know more about our users. It would be very helpful if you would answer a couple of questions in this survey. The only data that we gather is directly what you say in the survey - no IP address, browser information etc. is collected or stored at any point. No question is mandatory - if you would like to only answer one or two, it will still be a lot of help to us!
1, Which languages would you like to use Qubes in: [multiple choice]
What is the main reason you use/want to use Qubes?
Are you: [multiple choice]
Do you prefer managing your system with:
What other operating systems do you use:
Do you want to use Qubes on:
If you are already using Qubes, how many custom qubes (VMs) do you have:
If you are already using Qubes, how many qubes (VMs) do you usually have running at the same time, excluding the system qubes like sys-net or sys-usb:
If you are already using Qubes, which templates do you use:
@marmarek , @ninavizz , do you have things you'd like to ask?
2. What is the main reason you use/want to use Qubes?
Increased security
Ease of compartmentalizing different parts of your computer use
I'd split the second option into two cases, like:
@marta I love that you started this issue and have given this so much thought so far, thank you!! <3
TL;DR, "Who are Qubes Users?" is a major project I have scoped for a funding-ask that is in review, right now. SimplySecure has their own instance of Lime Survey, hosted on a server in Germany, that I'd thought to use for a variety of Qubes surveys—this, among them.
That said, getting funds approved so I can properly begin work on this stuff, has taken a lot longer than I'd expected... and I have been in convos with Georgia to get me access to their Lime instance sooner rather than later, so we can start on this one. I apologize for not being in communication with y'all about my ongoing work on this stuff for Qubes—this funny respiratory virus has kind of been taking over the world, and upended other projects I've had to focus on re-planning... #distracitongalore
What have your thoughts been for implementation? Yes, I totally have ideas for this one, including branched logic drill-downs. Would love to do an initial "UX Meeting" to chat through some of these things, so we could launch something in time for (or sooner than) the 4.1
release.
I was thinking about Lime Survey, actually - setting our own up, but using an existing instance would be even better and easier.
And no worries - I think everybody's world is askew these days and nobody expects anything to go quite as planned :)
So, the SimSec instance is self-hosted on an Azure cloud server. Given this, and the paranoia/opsec-needs of many Qubes users, Qubes setting one up on their own server would not be a bad idea, perhaps?
Its configuration UI (for researchers) is a nightmare, from what I'm told—I don't know anything about hosting/running an instance.
I could go either way, TBH, and feel the server-security needs of Qubes users should be the motivating priority.
Tor does a nice job with a stand-alone page for all surveys. Doing something similar for Qubes, with a disclosure about how data is stored, what (if any) tracking tools are used on those pages, how the survey is hosted, and what is to be done with all survey data, feels like a good way to go about things.
For the "New User" survey, we'd then link to the above page from the Downloads page, with an appropriate link to the correct survey in a clearly actionable fashion.
@marmarta ^^ comment edited to reflect deets on SimSec's hosted Lime instance.
Also, to prevent a never-ending thread of comments-as-build, I've created an Etherpad here for crafting our survey.
@hexagonrecursion There are established best-practies in how to word questions to get the best results... and, also how to frame answers as to segment sought learning-interests as best as possible.
I personally find it easiest to work-backwards... sketch-out a survey, first, then step back to ask what I really am seeking to learn, conceptually/strategically... then move forward again, with that focus in mind. Then, fine-tune per survey best practice standards.
For sketching the initial survey, it's probably best to not fuss w/ language so much. Also, it really helps to discuss as a team synchronously, at some point. Language is so fussy and subjective, and if folks have ideas for what they want to learn about but don't know quite what questions would align, it'd be great to make space to explore that territory, too.
Also, it just shouldn't be too long—no survey should take more than 5min, unless it's a follow-up survey with an invested audience. So establishing clear goals against which edits get made, will help a lot. :)
If folks are comfortable naming themselves in survey edits, suggestions, or discussion, that—and picking a nicely contrasting color against the others—would help a lot in the Etherpad draft! :)
Thx to whomever added to it already.
Hmm, OkThanks created an OONI survey using Typeform. That's curious! I'll ask some Tor folks how they feel about use of that platform. At a minimum, it could be nice to just solicit some basic insights on Qubes users, there, in advance of a more permanent tool (such as Lime) being setup.
https://ooni.org/post/2020-ooni-run-survey-and-interviews/
^^ @marmarta
Another thing we'd like to know: preferred desktop environments - both those already supported and those not yet. Provisional list:
Another thing we'd like to know: preferred desktop environments - both those already supported and those not yet.
Agreed. Also...with 4K displays, the possible additional layer of window-content buffering (i.e. with the currently-optional GUI domain), and the lack of application-reachable GPU acceleration in Qubes by design...video/graphics performance will continue to be a key issue w/r/t desktop environment selection.
Targeting 09 Sept Go Live
- Outreach Channels
- Mailing list(s)
- Bi-Weekly reminder blasts for 2mos
- Download page on website
- Discord
FYI, the forum software is "Discourse," not "Discord" (similar names; the latter is a different popular communication platform).
I have a system for handling our announcements so that they're disseminated consistently across all of our communication channels (some of which are not listed here). I'd be happy to put this through the pipeline so it gets the full treatment. Just let me know when it's time. I'm also available to help with drafting the announcement, if desired.
@andrewdavidwong I'd love to know what channels are not listed here, as different channels will get different text composed to promote engagement. There are established best practices around composing text to get users to engage with, so I'm cool doing a first pass on all—but always love editing to my text, which sometimes can get longwinded. :)
Second, and more urgent—I'm not seeing any Google Analytics codes embedded into the pages of the website. Do y'all do any analytics to understand how users navigate the website?
I'd like to remove the blurb about donating to Qubes, from the Downloads page, and replace it with a blurb promoting the survey... namely, because I don't want to add more content above the fold, as that page already has potentially too much. However, if that's a major engagement point for donations, it'd suck to remove it. If you could let me know on this asap Andrew, I'd muio appreciate! We're hoping to go live with the survey in a few days. Andrew, I'll also send you updated text to put in that area if you and @marmarek are ok with replacing that content with a call to do the survey.
Longer-term, however, I'd also like to do a quick update to the contribute/donate experience—as today it seems unclear to people how to contribute to Qubes as a developer, and "donate" is lost among other header stuff.
@andrewdavidwong I'd love to know what channels are not listed here, as different channels will get different text composed to promote engagement. There are established best practices around composing text to get users to engage with, so I'm cool doing a first pass on all—but always love editing to my text, which sometimes can get longwinded. :)
Certainly, here's the full list:
The new Qubes forum automatically pulls in new items from Qubes News. Usually, I publish a post on Qubes News, then simply share the link on the social media sites. For this, we can instead do tailored posts for some or all of the platforms, if you prefer.
Second, and more urgent—I'm not seeing any Google Analytics codes embedded into the pages of the website. Do y'all do any analytics to understand how users navigate the website?
No, and that is very much intentional. Our userbase is exceptionally privacy-conscious (as are we). Not only would using Google Analytics go against our principles; we would also immediately lose credibility with our core demographic.
I'd like to remove the blurb about donating to Qubes, from the Downloads page, and replace it with a blurb promoting the survey... namely, because I don't want to add more content above the fold, as that page already has potentially too much. However, if that's a major engagement point for donations, it'd suck to remove it. If you could let me know on this asap Andrew, I'd muio appreciate! We're hoping to go live with the survey in a few days. Andrew, I'll also send you updated text to put in that area if you and @marmarek are ok with replacing that content with a call to do the survey.
I understand. We don't know how many clicks that Donation button receives. However, one option would be to remove the top row of links (the row above the donation row) in order to make room for the survey. Another idea (which could be combined with the first) is to put the survey adjacent to the donation blub so that they're side-by-side. (On very small screens, we may still have to give each its own row.)
Longer-term, however, I'd also like to do a quick update to the contribute/donate experience—as today it seems unclear to people how to contribute to Qubes as a developer, and "donate" is lost among other header stuff.
Sounds good. We do have Contributing, but I'm sure it can be improved.
Thx for the quick response, @andrewdavidwong!
The top row of links feels very important, so I don't want to touch it. Stuffing yet another piece of information on that page is certainly possible, but will muddle the volume of information already present. Clear hierarchies w/o too many options, is important. I'm hesitant to add a second column of content where the "Donate" text is, atm, because that would put two things at the same visual hierarchy. I'd honestly rather just replace that content right now, with a call for the survey—and then in a week or two add a uniquely designed module to solicit donations.
"Contributing" is not easily discoverable on the website. Users need a top-level something to go to, where two binary options are clear: "give us money," or "give us your time." I'm fine tackling that as a simple IA challenge with some tweeks to the website header, sometime next week. Not a hard design challenge, but I won't be able to code it—so someone else wd need to volunteer to do that.
The top row of links feels very important, so I don't want to touch it.
Many of them are important, but many of them are also duplicated on each release row (to make it more likely users will see them).
Stuffing yet another piece of information on that page is certainly possible, but will muddle the volume of information already present. Clear hierarchies w/o too many options, is important. I'm hesitant to add a second column of content where the "Donate" text is, atm, because that would put two things at the same visual hierarchy. I'd honestly rather just replace that content right now, with a call for the survey—and then in a week or two add a uniquely designed module to solicit donations.
Ok, that's fine by me.
"Contributing" is not easily discoverable on the website. Users need a top-level something to go to, where two binary options are clear: "give us money," or "give us your time." I'm fine tackling that as a simple IA challenge with some tweeks to the website header, sometime next week. Not a hard design challenge, but I won't be able to code it—so someone else wd need to volunteer to do that.
Ok, sounds good. Depending on the type of coding involved, I might be able to help (e.g., if it just involves modifying the website).
Just discovered a to-do for when the LimeSurvey instance is depreciated to a more stable 3.x version... there is a global setting whereby all answers can have "No Answer" added to all questions, by default. Which I love, because it places that option below the "Other:" option—which is important.
That said, this the "No Answer" option is hard-coded into LimeSurvey to display the No answer
language to users... and, for it to always be selected by default. I dislike the terse "No answer" language.
So, this comment's request
Prefer not to answer
from No answer
Below is a screenshot showing where it is in the code.
@marmarta So... I'm an excellent hack at forcing CSS and HTML to bend to my will in a local browser, but I stink at making live code... umm, like, work. It doesn't seem like rocket science, but despite access to all the frontend stuff through the Admin part of the Lime UI, it's something I'm not able to get right. As such, I wanted to ask if you might be keen to take a spin at pushing the below tweaks? I was able to update the survey header logo, at least!
The above comment's to-do item also needs doing, but I'm ok with the "No Response" language if you need to cut scope.
If you're keen to take a spin at my failed effort on the styling updates...
.group-title {
font-size: 3em;
font-weight: 300;
text-align: left;
margin-top: .1em;
}
.group-description {
padding-left: .8em;
font-size: 1.5em;
color: #999;
font-weight: 300;
}
.question-container {
border: 1px solid #f3f3f3;
background-color: #fafafa;
}
Delete all below values associated with page's "Well" style, which is the goofy box behind the group description
.well {
/* min-height: 20px; */
/* padding: 19px; */
/* margin-bottom: 20px; */
/* background-color: #f5f5f5; */
/* border: 1px solid #e3e3e3; */
/* border-radius: 4px; */
/* -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05); */
/* box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05); */
}
These evil blocks of CSS from the WYSIWYG editors in the admin UI
#docs-internal-guid-09d51d1f-7fff-ae48-62db-c8a514bbcf40 {
/* font-size: 10pt; */
/* font-family: 'Proxima Nova', sans-serif; */
/* color: #000000; */
/* background-color: transparent; */
/* font-weight: 400; */
/* font-style: italic; */
/* font-variant: normal; */
/* text-decoration: none; */
/* vertical-align: baseline; */
/* white-space: pre-wrap; */
}
f05a
in the FontAwesome library.
.fa-exclamation-circle::before {
content: "\f05a";
}
.ls-questionhelp::before { content: "\f05a"; }
.ls-question-help, .ls-questionhelp { margin-top: -5px; }
.dir-ltr .ls-questionhelp::before { left: -1.3em; }
<img width="1308" alt="Screen Shot 2020-09-29 at 11 37 08 PM" src="https://user-images.githubusercontent.com/8262612/94651196-ed12e100-02ac-11eb-86ba-2364648ffb0d.png">
## Welcome Page
- [x] Typesetting Survey Title
- [x] Typesetting Description
- [x] Typesetting Introduction
- [x] Kill styling from that awful WYSIWYG editor in the admin UI
- [x] With the WYSIWYG editor being killed, ensure the `limesurvey` and `survey@qubes-os.org` links remain live
- [x] Make page-title in HTML say "Qubes Survey," so the text on the Welcome page can say "Welcome!" (unless I can figure that out in the global settings on Lime, in the next 10min)
.survey-name { font-size: 3em; font-weight: 300; text-align: left; margin-top: .1em; color: #888; }
.survey-description { font-size: 1.5em; color: #333; font-weight: 500; text-align: left; }
.survey-welcome { font-size: 1em; color: #999; font-weight: 300; text-align: left; line-height: 1.5em; padding-top: .5em; }
@marmarta Last nina-ask before going to bed... might a PGP key be possible to offer users for the survey@qubes email? I saw SimplySecure do that for a survey that was just pushed for another project, and that seems inline with our user interests. Text is in the current version of the survey on line, with a placeholder # for a URL.
Ok!! 90% of the survey is done!
I am concerned about its length, so am pausing to ask folks in a new issue to please give it a spin in a timed capacity—to let me know how long it takes. I'd love to include all the free-text questions on the future of Qubes, but am concerned they may need to be condensed.
This is the GDoc with questions & outreach text. https://docs.google.com/document/d/1C71M2FkuwkdRwB6fcak_YTS9H3j_FE4OqXatxzqmdwk/edit?usp=sharing
Just made the survey. It's very nice! Thank your for that. Just two things (third for fun) from my humble noob point of view:
After taking the survey example for Issue https://github.com/QubesOS/qubes-issues/issues/6100 , I just wanted to offer some input on it.
Perhaps add an additional option such as running some applications through tor instead of just choosing between browser only and whole computer.
Maybe add an option for users who are more unfamiliar with different environments "Not familiar enough". To anyone moving from a Windows or MacOS environment, the first Linux Environment to them will be what they associate with "Linux" and in most cases they won't be aware that it can be changed or how to change it if it's not obvious.
Adding an extra option here like the last question would also be a good idea for metrics. Many new users may be unfamiliar with linux that they can't really give any input on which distro to use, but having that extra option, I think, would be very valuable so they're not just lumped in with "prefer not to answer".
Perhaps add an extra option here for "Rarely"
Also as a note, any question that has Checkboxes doesn't allow the user to check the box next to fields with inputs such as "Other" selections. It becomes checked automatically when the user types something, but it just seems odd when you click and it doesn't check, it just moves the selection to the input box. I assume this is just an issue with the survey provider. Radio buttons aren't affected since they become selected on click, but selecting them doesn't move to the input box in that case.
Thanks for the input, @zellchristensen ! I've added some of your comments (all except the "rarely" - I need to think a bit about it, as the whole sometimes/rarely/often etc. scales are fiddly and annoying things :) )
Just made the survey. It's very nice! Thank your for that. Just two things (third for fun) from my humble noob point of view:
* Tor question would probably deserves little example like mirror servers or such.
@fepitre thanks :) ! Could you perhaps try to write something? Neither me nor @ninavizz are super Tor pros, so we're a little apprehensive in how to word it not to sound bad :)
* I think I've answered two times the question about Desktop/Laptop
Yeah, one is about "general computer habits" and one is about "Qubes use"... maybe we should emphasize this a bit more.
* @marmarek's best fuzzer as closing picture (https://blog.marmarek.net/assets/fuzzy-tester.jpg) smile ?
I'm very much for that <3
Thank you @fepitre for the wonderful fuzzer-pic! Indeed, that had been my preference but I've felt like such a nag burdening Marek with too many things so far wit this project... the last thing I wanted to ask was, "Ohey, now that you've debugged this and that, pls send me a cat picture." DONE!!
Also, terrific feedback, TY!
@zellchristensen TY for the highly detailed (aka awesome) feedback! I've changed a setting on the survey so that question codes are now visible, to hopefully relieve the burden of screenshot-posting for folks. Yes, some of the questions text and logic-patterns need some smoothing, too. Much appreciated!
Should we caption the cat picture to say that it's Marek's cat? I wouldn't have known unless I had read this thread. :joy:
And another test performed:
Ok, to be fair, the kittehs have a mommy AND a daddy. Are we trying to keep that relation between the two human contributors on the downlow, or could we properly attribute?
Ok! I made a couple quick fixes to the survey, and relaunched it just now. Namely, I wanted to expose the question codes for folks to deliver clearer feedback (especially as things impact questions logic); as the above is all very helpful. Kitteh got swapped too, of course.
Today is supposed to be my tech-shabbat, so the rest I'll make tomorrow or Monday... pls keep the terrific feedback coming! :)
Ok, to be fair, the kittehs have a mommy AND a daddy. Are we trying to keep that relation between the two human contributors on the downlow, or could we properly attribute?
I thought about that too but didn't want to be presumptuous in assuming joint custody, so I just went with what @fepitre said. >.<
Oh no, this particular kitty has a clear preference, he is definitely @marmarek's cat (and his own fuzzy tester, because I'm much better at teaching kitties not to sit on my keyboard :D )
The purpose of this test survey is to learn about you: a current, former, or soon-to-be user of Qubes OS.
That language might put some people off. True, what you're doing here is learning about the users. But how that data will be acted upon is more impactful and may lead to more willingness to fill out the survey. Something along the lines of "the purpose is to ensure Qubes OS meets the needs of the community by knowing what the community seeks from Qubes OS". This line of thought may also help to ensure the questions asked are fulfilling a goal as opposed to just information collection.
A suggestion, if I may - make the questions more targeted towards usage of Qubes OS as opposed to generic "do you use this or that?" questions; I wouldn't put too much emphasis on general habits. I could exercise and workout every day but it still doesn't mean I will visit your gym until I know your gym fits my needs. One questions asks, "how often do you use Tor". If someone answers "Often", does that automatically mean they'd use Tor with Qubes OS? I would surmise this question seeks to determine how much effort should the Qubes team put into supporting Tor or Whonix-related issues, as they relate to Qubes OS. So why not just ask a binary "Would you use Qubes OS if it didn't include Tor/Whonix?"
Just my two cents, I hope the survey provides good data!
@icequbes1 I love your idea for tweaking the intro text, and will definitely edit it to that effect!!
The Survey live as it is right now, also does not include 5 open free-text questions for people to openly share their thoughts on Qubes OS. I needed to see how long the rest of the survey took, before adding those in as either their own separate survey, or a condensed free-text question, or as they are. Likewise, richer inquiry to learn what their experiences with Qubes have been like to date, needs to happen in follow-up interviews. Surveys are a poor vehicle for gathering that kind of data.
The primary goal with this survey, is to recruit folks we can follow-up with for verbal interviews and testing with prototypes. We need some base data about their use habits and such, to inform who to reach out to for interviews, though.
On your point with the Tor question... I want to see how often people use Tor and Whonix, regardless of their use of either on Qubes... and what they use Whonix for when on Qubes, to see how beneficial a simpler means to torify a connection might be to users, in Qubes. Getting those two data points isn't about drawing a conclusion, it's about deciding who to reach-out to for follow-up interviews to form an early hypothesis on that open question.
The "do you use this or that" questions are actually quite important, to understand the broader technology context that shapes users' expectations when they're using (or learning) QubesOS. Users cannot be asked "what do you want?" and provide answers that are true to what actually would make them the happiest. A small developer community might be able to do that, but Qubes also has to extend its reach to be financially viable over the long haul. I've been doing product design for 20 years; and users also often think they want something, that is actually very different from what tests well with them, later. Like, if you'd asked people what they'd want for transportation 150yrs ago, they'd say a "a faster horse to pull our carriage." Not a car. Today of course, we all say we want hovercars, but then ditch our IC cars for bicycles and public transit, lol.
We also need that context to evaluate how users participating in testing sessions respond to prototypes, down the road. Something I hope we can start doing, in 2-3 months.
Gotcha. Thanks for the additional context; I understand where you're coming from now.
And you're right - the hovercar I always wanted is still in the garage collecting dust!
@andrewdavidwong My internet has been down all day today... I'm hoping it can be resolved by tomorrow.
Any chance we could connect early this week to discuss website stuff? I'll throw together some mockups of options, this eve, and post, here. If so, let's connect via email to schedule something.
Sure! Sounds good.
When might you be able to work on implementing updates to the website to get the survey on it?
I can start immediately.
I'll also be working on drafts of the first 2 questions for the survey, today, and sharing them in the SimplySecure community, for other UX researchers to weigh in on, per Marta's concern (expressed in the gdoc) "Is this asking for feelings or objective skill measures?"
Sounds good.
A "Qubes Research Comms" Signal number sounds like it might be in order, for follow-ups to the survey... since folks are going to hopefully be responding with their Signal info. Same, for a "research@qubes-os.org" email address. Ideally I could have access to both, for scheduling and communication with research participants... but this is a post-launch/coordination thing I'd like to get on folks' radars...
Ok, I'm not the one with perms to set up a new qubes-os.org email address, but that sounds good to me.
@marmarek I need some answers for a question about Whonix. @marmarta Ok, I changed the first 2 questions into an array. The free-text questions are all in there, and to me this is mostly set. One "Tor" answer I think I need you to clarify your intentions with, but otherwise it looks good to me.
Care to give it a final(ish) spin? https://survey.qubes-os.org/index.php?r=survey/index&sid=137242&lang=en
Anyone else following: Last call for feedback! @mfc have you had a chance to review any of this?
@andrewdavidwong I will reach out to you this eve or tomorrow am. Need to go purge my brain...
I need some answers for a question about Whonix.
I can offer some basic info about this, in case it helps. Two of the main ways of using Whonix inside of Qubes are:
Using the Tor Browser in a Whonix Workstation qube in order to browse the web. This is also the most common use of Tor outside of Qubes, e.g., in non-Qubes Whonix, in Tails, or just using the Tor Browser Bundle on any mainstream OS. The advantage of doing this in Qubes is that the Tor Browser and Tor daemon run in separate VMs, so your browser getting compromised doesn't necessarily deanonymize you. The advantage of specifically using a Whonix Workstation qube is that the Whonix Workstation template takes special efforts to protect against VM fingerprinting and other advanced deanonymization attacks, along with other privacy and security enhancements.
Using sys-whonix
to download dom0 and template updates over Tor. This has some security advantages, potentially making it harder to be subject to targeted attacks via malicious or withheld packages. We offer an explicit option in the installer to set this up for the user.
Ok @andrewdavidwong everything is done from mine and @marmarta's end... GDoc is current with website and announcements stuff. Once those are completed, this issue may be closed from my end! Marta may want to also add some completion criteria for this to close, too.
Welcome!
page.@marmarek or @andrewdavidwong create key for survey@ email address
I can easily generate a new PGP key, but I still can't create new email addresses, so it probably doesn't make sense for me to make the key.
@ninavizz, @marmarta: Two users have reported this problem:
the 'next' button just reloads the page, no progress for me. Firefox Android.
interesting, after updating my phone firefox I got the same result... I'm afraid this is LimeSurvey bug.
Thanks for flagging and testing, y'all! Added a note on the first page. It's horsey, but it works. :/
Perhaps add the line of text to the blog article, after the period on the sentence just before the link: Of note: we've had reports that in Firefox on Android devices, it does not work. FYI.
Thoughts? @andrewdavidwong
Perhaps add the line of text to the blog article, after the period on the sentence just before the link:
Of note: we've had reports that in Firefox on Android devices, it does not work. FYI.
Thoughts? @andrewdavidwong
Added!
A user is reporting issues as well with firefox but this time on a debian-10 qube. I was unable to reproduce it.
I can’t proceed even to the first question, it drops me back to 0% percent every time… is it closed now ? (using debian-10 qube & firefox) -- user JBence on this forum post
Brief summary We have very little information about our users. It would be really helpful from design point of view to know a little better who Qubes users are, what background (like, other OSes they use) they come from and what they need.
As we will not spy on users in any way, I think a decent method would be making a questionnaire (not google forms obviously) that would be shown as "would you like to answer a few questions" when someone downloads Qubes ISOs.
Of course no IPs and any information other than what's voluntarily given would be collected.
Questions I propose: