search

haskellweekly / haskellweekly.github.io

A free weekly newsletter about the Haskell programming language.
https://haskellweekly.news
MIT License
89 stars 33 forks source link

2018 state of Haskell survey #206

Closed tfausak closed 6 years ago

tfausak commented 6 years ago

📢 You can preview the survey here: https://airtable.com/shr8G4RBPD9T6tnDf. 👂 Let me know if there are any questions that you would like to be added, removed, or changed by replying to this GitHub issue. Thanks!

I want to get the ball rolling on this year's community survey.

👀 #102 #121 #125

tfausak commented 6 years ago

Now that November 1 is less than a month away, I think it's time to actually start putting this together. Broadly speaking I think the survey's questions will be similar to last year's. In addition to the links above, here are some valuable resources:

I have been using Airtable for other projects and really enjoy it. I will probably set up this survey to be run through Airtable rather than the static HTML and Formspree setup I used last year.

tfausak commented 6 years ago

I started translating the old survey into Airtable. You can see it here: https://airtable.com/shrigdVbhGA5NH60E https://airtable.com/shr8G4RBPD9T6tnDf

lettier commented 6 years ago

Hello @tfausak,

I may have a useful suggestion.

What are your thoughts on adding a multiple choice question regarding what Haskell related content topics the respondent would enjoy reading? This could help discover a gap in coverage, if one exists, and/or help Haskell content creators align their efforts with the needs of the greater Haskell audience. You could even use the results to help decide what to feature in future Haskell Weekly issues. :)

:+1:

tfausak commented 6 years ago

That's a great idea! Do you have something in mind for the question itself and the answer choices?

lettier commented 6 years ago

For the choices, we could take a data driven approach. Using topic analysis on the linked to articles, I can try to uncover the hidden themes found in all of the back issues of Haskell Weekly.

Another approach would be to infer some choices from Answer the Public for the query "Haskell".

In it I see:

In addition to the specific options, it might be best to have a write-in option as well.

For the question, here are some possible choices:

Or, more interestingly, you could flip those:

It may be helpful to include both a positive and negative question using the same categorical options. They may be truly disjoint or have some significant overlap indicating controversial or "mixed bag" topics.

:+1:

tfausak commented 6 years ago

Thanks! That's a lot to chew on. Of the questions you listed, the ones I'm personally most interested in seeing the response to are "Which of the following Haskell topics do you want to see more/less written about?" Those would be very useful to me as an author/curator.

tfausak commented 6 years ago

Note to self: Areas not covered by last year's survey that I want to cover this year:

Other useful links:

kitoran commented 6 years ago

Hello @tfausak. I'm sorry if this isn't the right place to express my sugestion, but here we go. In the results of 2017 state of Haskell survey results there are questions concernig satisfaction with the language and tools: I am satisfied with Haskell as a language. I am satisfied with Haskell’s build tools (such as cabal-install or Stack). I am satisfied with Haskell’s package repositories (such as Hackage or Stackage). I think it's useful to add similar question about compiler - I am satisfied with compiler that i use.

tfausak commented 6 years ago

This is definitely the right place, and thanks for the suggestion! I would be happy to add a question about compiler satisfaction.

tfausak commented 6 years ago

I'm not very happy with how Airtable is working out as a form solution. A few things in particular are annoying me:

  1. It's not possible to have a single question with multiple answer choices along with an "other" choice. Since this is something that I do a lot, it's really frustrating.

  2. It's also not possible to show/hide a question based on the answer to a previous question. Many of the questions are follow-ups for certain answers, and it would be nice to hide them.

  3. It's not possible to break things up into sections, which I think would help make the survey feel less imposing.

I think Airtable could be a good resource for analyzing the responses, but I'm going to investigate other form services.

tfausak commented 6 years ago

Another thing I would like to ask about is Haskell Prime, also known as Haskell2020, or just the next version of the Haskell standard. Do people want it? What do they want included in it or excluded from it?

tfausak commented 6 years ago

Discussion is also happening on the haskell-community mailing list: https://mail.haskell.org/pipermail/haskell-community/2018-October/000323.html

tfausak commented 6 years ago

After briefly looking at SurveyMonkey and Google Forms, I think I'll stick with Airtable. It's pretty bare bones, but it'll get the job done.

I plan on working on the survey this weekend and publishing it on Thursday. If you have any feedback about this year's survey, now's the time!

ChrisPenner commented 6 years ago

I haven't fully thought through this yet, but I thought the way that the state of JS survey displayed some of their results was really cool! They allow you to see connections between tools and find pockets of users; e.g. Maybe most of the users of nix also use cabal instead of stack; etc. Not sure how tough it would be to do something like this, but it's definitely worth a look!

https://2017.stateofjs.com/2017/connections/

mboes commented 6 years ago

Would there be any way of of developing the survey questions in GitHub? One idea would be to do it as a Markdown document, then convert that to a Google Form or similar at the last minute. That way people can make PR's to suggest improvements.

tfausak commented 6 years ago

@ChrisPenner: That is really cool! I feel like I lack the skills to make something like that. I will be releasing the survey responses, so hopefully one of the more talented members of our community can produce something similar.

@mboes: That's a great idea! Unfortunately I don't think I can support that workflow for this year's survey. It sounds awesome for next year, though!

ChrisPenner commented 6 years ago

@tfausak No prob; I can probably whip that up when the data is released 😄 Looks like it's just a matter of dumping the data into this thingy http://nivo.rocks/chord/

emhoracek commented 6 years ago

Please also ask about gender. I feel like I'm an extreme minority as a woman in Haskell -- whenever someone says they know a woman in Haskell, it's only of the <20 other women I already know in Haskell. :/ I have a feeling there are more of us out there who just aren't as public, so it would be nice to know that, and to see if in future years there are more.

Also, if you do ask about gender, please use an open field, or follow these guidelines: https://www.hrc.org/resources/collecting-transgender-inclusive-gender-data-in-workplace-and-other-surveys

Thanks!!

tfausak commented 6 years ago

Thanks for the suggestion, @emhoracek! I previously added a demographic question to the survey, which I cribbed from Rust's survey. Here it is:

Do you consider yourself a member of an underrepresented demographic in technology? (multiple select)

  • Cultural beliefs
  • Disabled or person with disability
  • Educational background
  • Language
  • Lesbian, gay, bisexual, queer, or otherwise non-heterosexual
  • Non-binary gender
  • Older or younger than the average developers I know
  • Political beliefs
  • Racial or ethnic minority
  • Religious beliefs
  • Trans
  • Woman or perceived as a woman
  • Yes, but I prefer not to specify

I understand that is broader than gender, but do you feel that it covers gender? I could add another question asking about gender specifically if the above question isn't sufficient.

emhoracek commented 6 years ago

@tfausak I think that's a pretty good question, and the number of people who select "woman or perceived as a woman" would provide the information I want. Do the other options provide useful information though? If many people select "Older or younger than the average developers I know", you don't know if that means Haskell has plenty older developers or plenty younger developers, or it's split and there's actually not that many older or younger developers. I would at least split that option into "older than the average developers I know" and "younger than the average developers I know". Educational background could be similar except in Haskell I doubt that you're going to have folks with PHds saying they're underrepresented. :)

tfausak commented 6 years ago

Thanks, that's a good point. I've been looking over the Stack Overflow survey results and I like how they split out all the demographic questions. I think I'll do something similar.

I've also been looking at the Go survey results and noticed that they asked a lot of good "I feel ..." type questions. I asked many of these last year, and I feel like they were the most valuable things to me. (Hopefully others found them valuable too.) This year I'd like to ask many more. These are the questions that the Go survey asked:

tfausak commented 6 years ago

I'm trying to distill the haskell-community mailing list thread down into an actionable list of things that I can do tomorrow. Here's what I've got:

I think I got everything else covered in this comment: https://github.com/haskellweekly/haskellweekly.github.io/issues/206#issuecomment-429641438. Copied here for convenience:

And then all of the questions from my previous comment, taken from the Go survey. And then these questions, takes from various comments in this thread:

There's some overlap between those things. Today I've been cleaning things up and adding demographic questions. For some reason I decided to do this on a new form rather than the existing one. You can find it here: https://airtable.com/shr8G4RBPD9T6tnDf

tfausak commented 6 years ago

Alright, I think I've got the survey in the shape that I want it. I've included as many of the desired questions as I felt like I could. It's hard to balance adding questions that could be useful with keeping the survey short so that people will actually respond to it.

Please take a look at the survey to make sure that you're happy with it! Let me know if there are any questions that you would like to be added, removed, or changed. You can view the survey here: https://airtable.com/shr8G4RBPD9T6tnDf

soupi commented 6 years ago

Maybe a few questions on education? For example, How did you learn Haskell? How long did it took until you built your first project? Would you recommend it to others? Etc.

tfausak commented 6 years ago

Those are good suggestions! I'm interested in asking them, but I'm not sure what the answer choices should be. I'm also not sure if the results would be actionable.

soupi commented 6 years ago

So the main thing I'd like to understand is what is the state of learning Haskell for a beginner nowadays. How quickly can a person get up to speed with Haskell? Is self study with a book enough? How quickly can they start making their own projects? Which resources work best? Are we still in the need to improve the state of learning Haskell?

This kind of information is useful for haskell advocates (can I recommend learning haskell to a colleague or a friend?), to hiring managers (can I get people up to speed fast?), people that might consider improving the learning curve (is there more room to improvement or are we good enough?), etc.

I hope this makes more sense. If you don't feel like asking this in this survey that's ok too!

tfausak commented 6 years ago

Sorry for the late reply, @soupi! I would like to ask some questions about learning Haskell. You suggested a lot of questions, and I am interested in the answers to all of them. However I'm not too sure about adding so many questions to the survey — it's already longer than I would like. If you could only pick one or two questions (preferably multiple-choice), which ones would you pick?

tfausak commented 6 years ago

Unfortunately the deadline got the best of me this time. We'll have to wait until next year to include questions about learning Haskell in the survey. I'm sorry.

soupi commented 6 years ago

That's fine. Good luck with the survey!

tfausak commented 6 years ago

Thanks!

The survey is now open. You can read the announcement post here: https://taylor.fausak.me/2018/11/01/2018-state-of-haskell-survey/

You can go directly to the survey here: https://bit.ly/haskell2018