search

nodejs / next-10

Repository for discussion on strategic directions for next 10 years of Node.js
Other
479 stars 24 forks source link

April 2024 Survey #261

Closed marco-ippolito closed 4 months ago

marco-ippolito commented 6 months ago

Fixes: https://github.com/nodejs/next-10/issues/250 I've copy pasted 2023 survey, feel free to add/remove question

kyliewd commented 6 months ago

Thanks @marco-ippolito @mhdawson. I have this started, but I think its a bit too long and users might not complete the full survey. Is there any questions that can be removed? It's best to have 10-12 questions maximum to get the best response.

AugustinMauroy commented 6 months ago

I know that the discussion is tense on this subject. But we should have some stats/question on Corepack.

for me the two questions that are missing:

mhdawson commented 6 months ago

We should also look at the results from the last survey. Even though overall the answers/data was great, I know there were a few questions were the answers suggested the question was not very clear. We should identify those and make sure to tweak the related questions.

kyliewd commented 6 months ago

i just went through last year's survey data and i recommend the following:

kyliewd commented 6 months ago

@mhdawson @marco-ippolito here is what i have right now: https://www.surveymonkey.com/r/Preview/?sm=azuRRBptcGtNPJ1HXU5pgFGqa9o53iCd8PmuDc0c_2FMfpPH9Tqx_2FPQV1lAeZ7BYY7

please note that i don't think we should have open ended questions. this doesn't guarantee a good answer rate. recommend that if there is one i didn't add due to that, we make it multiple choice if possible.

mhdawson commented 6 months ago

no open ended questions (many people skipped)

I agree we should limit these, but we got a lot of very interesting responses in the last survey (several hundred) even if many people skipped (since we got a lot of responses).

GeoffreyBooth commented 5 months ago

I remember digging through the feedback from the last one, to try to identify biggest pain points: https://github.com/nodejs/TSC/issues/1445#issuecomment-1745405198. It would be nice if we can somehow gather this information in a more structured way than a free response “what issues are you having with Node.js” open question.

It was clear from the last survey’s data that many users struggle with ESM and TypeScript, and that’s probably our biggest user complaint. What’s less clear is why they struggle, which is what we need to know in order to help them. Anecdotally it seems that the largest group of such struggling users are ones building React apps with Webpack, implying that it’s something about older build tools not keeping current with modern APIs; but I don’t really know much more than that. In a perfect world I’d somehow be able to get 100 responses from people who struggle with TypeScript and ESM, and they could be categorized into groups where these 50 are on this particular tech stack and are getting this particular error; and these 30 are on this other one, etc. Then we could reproduce the problems, and try to find solutions for them; better error messages if nothing else. But I don’t know how to get anywhere close to getting data like this. Perhaps “Have you run into issues with TypeScript and/or ES modules, and if so, what errors did you encounter and what tech stack are you using?”

kyliewd commented 5 months ago

This is the current survey draft, which should reflect all of the comments above: https://www.surveymonkey.com/r/Preview/?sm=azuRRBptcGtNPJ1HXU5pgFGqa9o53iCd8PmuDc0c_2FMfpPH9Tqx_2FPQV1lAeZ7BYY7

marco-ippolito commented 5 months ago

I'll try to gather everyone's feedback and update the survey. Remember we are having a next 10 meeting Wed 17th April.

marco-ippolito commented 5 months ago

I tried to apply all the suggestions, PTAL 😅

styfle commented 5 months ago

@kyliewd The slider looks like its 0 to 100 instead of 0 to 10. I put it in the middle and its value is 50.

image
kyliewd commented 5 months ago

@marco-ippolito @mhdawson the survey is ready for review here: https://www.surveymonkey.com/r/Preview/?sm=azuRRBptcGtNPJ1HXU5pgFGqa9o53iCd8PmuDc0c_2FMfpPH9Tqx_2FPQV1lAeZ7BYY7

My feedback is that this is a bit too long, at 23 questions, we're looking at a completion rate of only 57%. It would be good to cut down if we can, but if not, it's important to understand that you may not receive answers on all of the questions.

GeoffreyBooth commented 5 months ago

It would be good to cut down if we can

I suggested cutting “Are you planning to convert an application to native ESM or use native ESM for a future application?”

marco-ippolito commented 5 months ago

It would be good to cut down if we can

I suggested cutting “Are you planning to convert an application to native ESM or use native ESM for a future application?”

It has been removed

marco-ippolito commented 5 months ago

Tried to remove/merge a few questions

marco-ippolito commented 5 months ago

@nodejs/tsc In today meeting the @nodejs/next-10 reviewed the survey and applied changes. PTAL As a reminder the survey is planned for next week @kyliewd we cut some questions and made open questions into bullet points to it should be much better

kyliewd commented 5 months ago

Final survey is here. This will go live on Tuesday, April 23. I'll plan to keep it open for a month.

After that, I will send the results out.

GeoffreyBooth commented 5 months ago

Questions 13 and 15 are duplicates. It's also missing the question about typescript.

marco-ippolito commented 5 months ago

@kyliewd I updated 16 and 12 also as @GeoffreyBooth mentioned 13 and 15 are duplicates

sheplu commented 5 months ago

Some comments. we are close :)

Question 5 and 7 seems to be the same Question 9 should be a multiple choice - and do we want to keep all sub example with docker? Or just a global docker ? @marco-ippolito Question 10, change ia32 by x32 Question 20 should be a multiple choice Question 21 should be a multiple choice

kyliewd commented 5 months ago

I have updated all of these. Please have all final edits in by end of day today, april 22.

marco-ippolito commented 5 months ago

I have updated all of these. Please have all final edits in by end of day today, april 22.

Can you provide the link of the updated survey?

kyliewd commented 5 months ago

I have updated all of these. Please have all final edits in by end of day today, april 22.

Can you provide the link of the updated survey?

Same link as before

GeoffreyBooth commented 5 months ago

Same link as before

On that link, two questions both ask “How do you manage the package manager for your project?” We don’t need two copies of this.

I’d like this question added please:

If you use or have attempted to use TypeScript with ES module syntax, whether ESM dependencies or running your production code using ESM syntax, what issues (if any) have you encountered? Please be as specific as possible and please include your tech stack (build tool, application framework, testing framework, runtime customizations, etc.).

kyliewd commented 5 months ago

Same link as before

On that link, two questions both ask “How do you manage the package manager for your project?” We don’t need two copies of this.

I’d like this question added please:

If you use or have attempted to use TypeScript with ES module syntax, whether ESM dependencies or running your production code using ESM syntax, what issues (if any) have you encountered? Please be as specific as possible and please include your tech stack (build tool, application framework, testing framework, runtime customizations, etc.).

The duplicate was fixed. I will let @marco-ippolito and @mhdawson chime in but i strongly recommend that we don't have that broad of an open question + we cannot add another question, as we are well over the recommended limit

GeoffreyBooth commented 5 months ago

I suggest we find some other questions to cut, then, because I really need this information and we’ve been discussing it for the last few years, and the survey is the way it was suggested that we get this data.

mhdawson commented 5 months ago

we cannot add another question, as we are well over the recommended limit

@kyliewd, is this a hard limit of the tool.

If we just add the open ended question at the end of the survey, that should not affect the likelyhood that people don't complete the questions up till then right?

kyliewd commented 5 months ago

we cannot add another question, as we are well over the recommended limit

@kyliewd, is this a hard limit of the tool.

If we just add the open ended question at the end of the survey, that should not affect the likelyhood that people don't complete the questions up till then right?

Not a hard limit but you really start to lose folks after question 10 - so you're at risk for that. my goal is to help you all get the best response / completion rate here! 😄

kyliewd commented 5 months ago

@mhdawson if we need to add the question suggestion from @GeoffreyBooth it needs to be shortened/simplified.

GeoffreyBooth commented 5 months ago

Here’s a shortened version:

Have you encountered any issues writing TypeScript where the production code used ES module syntax? If so, please include as much detail as possible such as tech stack.

kyliewd commented 5 months ago

Survey is live: https://linuxfoundation.surveymonkey.com/r/nodenext10survey24

styfle commented 5 months ago

We make the distinction between local OS vs production OS, but then the next question for architecture does not make that distinction.

I think we need it there too. For example, macOS and arm for local vs Linux and x84 for production.

TLDR

Change this question:

Into two questions:

wesleytodd commented 5 months ago

https://x.com/justinfagnani/status/1783231031396721131

I thought this was good feedback. Is this something we can change while it is inflight?

mhdawson commented 5 months ago

@kyliewd thanks for all of your help on the survey so far.

kyliewd commented 5 months ago

This survey is already out in the field and gathering responses. We will not be making anymore changes as that messes with the data.

wesleytodd commented 5 months ago

~Cool, I assume though fixing the missing "USA" would not mess with the data? Just making sure both of their feedback bits landed.~ EDIT: just checked, which I should have done before posting and I see it is already addressed. Great job @kyliewd! Sorry for the noise.

kyliewd commented 5 months ago

EOW Update: 494 responses. I will update y'all weekly on the numbers. Keep promoting / telling your communities.

The survey will close on May 24.

kyliewd commented 5 months ago

Survey update: We have 1,612 responses. Please take the survey + tell your collaborator friends!

YamiOdymel commented 5 months ago

I told my collaborator friends and they said Taiwan and Hong Kong are not in the country list 💀

GeoffreyBooth commented 5 months ago

I told my collaborator friends and they said Taiwan and Hong Kong are not in the country list 💀

And yet the list includes some questionable options:

The list in the survey appears to be derived from this list of all countries with which the United States has ever had diplomatic relations, which would explain why an earlier version excluded the United States. A better list might be ISO 3166: https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes

kyliewd commented 5 months ago

I told my collaborator friends and they said Taiwan and Hong Kong are not in the country list 💀

And yet the list includes some questionable options:

  • Austrian Empire
  • Brunswick and Lüneburg
  • Central American Federation
  • Congo Free State
  • Czechoslovakia
  • Duchy of Parma
  • East Germany (German Democratic Republic)
  • Grand Duchy of Tuscany
  • Hanover
  • Hanseatic Republics
  • Hesse
  • Kingdom of Serbia/Yugoslavia
  • Lew Chew
  • Mecklenburg-Schwerin
  • Mecklenburg-Strelitz
  • Nassau
  • North German Confederation
  • North German Union
  • Oldenburg*
  • Orange Free State
  • Papal States
  • Piedmont-Sardinia
  • Republic of Genoa
  • Schaumburg-Lippe
  • Two Sicilies
  • Union of Soviet Socialist Republics
  • Württemberg

The list in the survey appears to be derived from this list of all countries with which the United States has ever had diplomatic relations, which would explain why an earlier version of excluded the United States. A better list might be ISO 3166: https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes

This has been adjusted.

kyliewd commented 4 months ago

we are at 2,312 responses for the survey. 🎉 A little less than 2 weeks until the survey closes. Thanks for your promotion and help here.

kyliewd commented 4 months ago

survey closes this friday! we're at 2,731 responses 🔥

mhdawson commented 4 months ago

Survey went out and closed, so landing this PR

joyeecheung commented 3 months ago

Are the survey data available yet?

mhdawson commented 3 months ago

We just finished the review/sanitization of the open ended questions and have the ok from the Foundation to add to the repo. @marco-ippolito were you going to do that?

marco-ippolito commented 3 months ago

We just finished the review/sanitization of the open ended questions and have the ok from the Foundation to add to the repo. @marco-ippolito were you going to do that?

Yes I'll open a PR