Open atopal opened 1 year ago
For context, this is also related to #15 which is where we started discussing about tracking, over time, the top pain points that web devs experience.
At first sight, it seems to me like devSat does this very nicely, and I would hate to have to replicate this. If Google already runs this on a regular basis, and is willing to consider sharing the results (also to the demographic questions), I don't think there's a need for the WebDX group to try to reinvent something.
Some random comments about the questions:
8 Automation of frequently performed small tasks
I'm not sure how this would be used. What is the intent of this question?
14 Supporting old browsers
Just a thought: now that IE11 is behind us, it might be nice asking a follow-up question for people who select this, asking about which older browser they find difficult to support.
12 Other challenges; enter here: [OTHER: M]
This is great to see. If new patterns start emerging over time within this field, then that could influence the list of questions.
Q2. Please tell us why you are with the Web, as a platform and set of tools, to enable you to build what you need or want.
I note I find it hard to grasp what "why you are with the Web" means. Is that "why you use the Web", "why you picked up the Web", something else?
"Why do you choose the web platform?"
@captainbrosset
8 Automation of frequently performed small tasks I'm not sure how this would be used. What is the intent of this question?
This is not so much a web platform pain point, but more of a dev experience issues with workflows. Do you mean it's unclear how we'd address issues?
14 Supporting old browsers Just a thought: now that IE11 is behind us, it might be nice asking a follow-up question for people who select this, asking about which older browser they find difficult to support.
Great point!
@tidoust
Q2. Please tell us why you are with the Web, as a platform and set of tools, to enable you to build what you need or want. I note I find it hard to grasp what "why you are with the Web" means. Is that "why you use the Web", "why you picked up the Web", something else?
Yes, as @tomayac said, it's "Why do you choose the web platform?" as a dev, but I agree, it's not straight forward, I'll bring that up as feedback.
The screening questions have a lot of logic, but I'll follow up with them shortly
In what country do you currently work? Select one.
1 Canada 2 United States 3 Brazil 4 Germany 5 France 6 India 7 Japan 8 United Kingdom 9 Spain 10 Mexico 11 Russia 12 China 14 Indonesia 13 Somewhere else
Are you currently…? Select one. If more than one answer applies, select the one closest to the top of the list.
1 Working as a full-time employee 2 Working as a part-time employee 3 Working as a contractor/on a temporary assignment 4 Working at a company or business I own 5 Working as a freelancer 6 Working as a consultant 7 Retired 8 A student 9 None of the above REF Prefer not to answer
Show if (JOB_TYPE=2): Does your company…? Show if (JOB_TYPE=1): Do you…?
1 Develop products for end consumers 2 Develop products for other companies 3 Develop products for both end consumers and other companies
Show if (JOB_TYPE=2): Does your company provide development services, including web development, to other organizations? Show if (JOB_TYPE=1): Do you provide development services, including web development, to other organizations?
1 Yes 2 No, we only develop in-house to meet our own needs
Show if (JOB_TYPE=2): Does the company/business you work at have a web experience?
Show if (JOB_TYPE=1): As a
1 Yes 2 No
Which–if any–of these describe your experience with software development? Select all that apply.
1 I write code and develop software on a regular basis 2 I manage or oversee software developers 3 I make decisions related to software development 4 I am a student 5 I am a teacher 6 Exclusive: None of the above
What is your level of experience in your area of work related to web development?
1 Expert: I can provide guidance, troubleshoot, and answer questions in my area of work 2 Advanced: I can perform the actions associated without assistance and am recognized within my immediate team as “a person to ask” when difficult questions arise 3 Intermediate: My focus is on applying and enhancing knowledge or skill. I require minimal guidance to perform successfully 4 Novice: My focus is on honing skills through on-the-job experience. I can understand and discuss terminology, principles, and issues related to my responsibilities 5 Fundamental Awareness: My focus is on learning and I have common knowledge or understanding of basic techniques or concepts 6 Not Applicable: I have not personally worked on development for any applications or solutions in my role
Is your main area of focus in your development…?
Web UI, CSS (Front of the front-end development) Web app/page functionality (back of the front-end development) Server side functionality (Back end development) Both front-end and back-end development (Full-stack development)
Over the past 12 months, which of these platforms have you developed for? Select all that apply.
Randomize
1 iOS (native apps/games) 2 Android (native apps/games) 5 macOS (native apps/games) 6 Social Platforms (e.g., Facebook, WeChat, Whatsapp) 7 The Web 9 Windows (WPF, UWP, Windows Forms, Win32) 10 Internet of things 11 ChromeOS 8 Others; list here: [OTHER: S] DK Don’t know
Is your general web development for…?
1 Desktop 2 Mobile 3 Both
The results of the questions have been posted here: https://github.com/web-platform-dx/developer-research/blob/main/vendor-research/Web%20Dev%20Sat%20H1%202023.pdf
Please use this issue for any further questions or comments regarding the survey questions, methodology, results, and possible changes for a future survey
In our last call, we discussed that it would be helpful to see the questions that will go into the Google DevSat survey. I'm sharing here what is currently planned to be asked in that survey. The actual survey has a lot of actual logic to filter based on answers, that I won't be able to replicate here, but I'm hoping the questions are illustrative of the survey itself. I have skipped the screening and demographic questions, but can also add those, if there is interest in that.
Questionnaire
Q1. How would you rate your overall satisfaction with the Web, as a platform and set of tools, to enable you to build what you need or want? Select the response that best describes how you feel.
Q2. Please tell us why you are with the Web, as a platform and set of tools, to enable you to build what you need or want. Enter your response in the box below. Be as specific as possible.
Q3. What are the biggest challenges you face while developing your web site or app? Please select up to three.
Ask those who find developer tools a challenge when developing for the Web Q4. What aspects of developer tools make it challenging when developing your web site or app? Select all that apply.
Ask if making a design/experience work the same across browsers is a challenge Q5. Why is making a design or experience work the same across browsers a challenge for you? Select all that apply.
Ask if has challenge with keeping up with changes to best practices for how to build for the web, with new features in web standards and/or with changes to existing APIs in web standards Q6. Why is keeping up with changes to the web platform/web standards a challenge for you?
Ask if testing end-to-end user flows or conducting other tests is a challenge Q7. What tests do you create and perform on a regular basis? Select all that apply.
Ask if testing end-to-end user flows is a challenge Q8. What are the challenges you face when testing end-to-end user flows? Select all that apply.
Ask if testing end-to-end user flows automatically is a challenge Q9. In what languages do you write your automated end-to-end user flows tests? Select all that apply.
Ask if testing across browsers is a challenge Q10. How do you perform cross-browser tests?
Ask if R tests cross-browsers manually only Q11. Why are you not automating cross-browser tests? Select all that apply.
Ask if R automates cross-browser tests Q12. What are the challenges you face when automating cross-browser tests? Select all that apply.
Ask if testing across browsers is a challenge Q13. Which types of features prevent you from making a design or experience work the same across browsers? Select all that apply.
Q14. What browsers do you support? For the purposes of this research, support means ‘build for and continue to support access on this browser’. Select all that apply.
Capabilities Section
Q15. How much do you agree or disagree with the following statement? All of the functionality I need to build my web app is available on the Web platform. Select the response that best describes how you feel.
Ask if R doesn’t agree that the Web platform has all the functionality needed to build their web app Q16. What user journeys within your web app have been hard or impossible to create due to missing Web platform capabilities? Enter your response in the box below. Be as specific as possible
Ad hoc section
Q17. Among these browsers, which ones would need to support a web feature before you can use it? Select all that apply.
Q18. For, how far back does it need to support a web feature before you can use it?