Devographics / surveys

YAML config files for the Devographics surveys
43 stars 8 forks source link

More objective metric for resources selection #52

Open SachaG opened 1 year ago

SachaG commented 1 year ago

Problem: current resources lists are too arbitrary.

Solutions:

Other possible metrics:

SachaG commented 1 year ago

Great resource to start with: https://www.classcentral.com/providers

perborgen commented 1 year ago

Hi @SachaG !

I think the "official list" is a bit arbitrary yes. However, combined with the "free form answers", it does cover most of the relevant JS learning resources on the web today. The only two additions I would include from ClassCentral are Coursera and Skillshare, as both are big on JS content.

As for the ranking, I think it's important to use an objective traffic measure. The best one is probably SimilarWeb's Global Traffic Rank (GTR). In this sheet, I have listed rankes the sites using the GTR: https://docs.google.com/spreadsheets/d/13PNP_t8ElHb8hoYZbcwQjn-jYzUTnbxh4isOe1D8n7Y/edit?usp=sharing

Note: for some sites, their Global Traffic Rank is too affected by non-JS-related-traffic in order to be relevant. I have moved these down towards the bottom (if they get voted to the top, then you can consider moving them up next year). The exceptions are Udemy and YouTube, as both clearly should be included in the "main list" IMHO, given their popularity.

SachaG commented 1 year ago

Super helpful, thanks!

SachaG commented 1 year ago

I think maybe we should just have a separate question for YouTube/Twitch actually, and free up one spot in this list.

SachaG commented 1 year ago

So here is the revised list, using a mix of the GTR rankings and the previous years' State of JS data. I left off YouTube because I want to give it its own category, and Reddit because I feel it doesn't quite fit the "learning" category as much as the "news" one.

Main List

      - id: stack_overflow
      - id: udemy
      - id: w3schools
      - id: mdn
      - id: freecodecamp
      - id: codecademy
      - id: web_dev
      - id: javascript_info
      - id: scrimba
      - id: frontendmasters
      - id: wes_bos
      - id: platzi

"Show More" List

      - id: skillshare
      - id: coursera
      - id: egghead
      - id: fireship
      - id: kent_c_dodds
      - id: o_reilly
      - id: pluralsight
      - id: linkedin_learning
      - id: pluralsight
      - id: designcode
      - id: academind
      - id: ui_dev
      - id: vue_mastery
      - id: newline
      - id: epic_react
      - id: midu_dev
      - id: levelup
      - id: doka
perborgen commented 1 year ago

Totally agree that it makes sense to move YouTube into its own category @SachaG . The revised list looks great IMO!

mor10 commented 1 year ago

I can only comment on LinkedIn Learning since that's what I have numbers for. We have front-end courses including CSS, JavaScript, HTML, React.js, etc with +300,000 active learners, and our front-end web library sees tens of thousands of active learners per month.

Some examples:

I agree for large platforms like ours, GTR is irrelevant, but that doesn't warrant relegating the platform to a secondary list. Instead, it means a different metric is needed. Most major learning platforms have public stats available on request and these could be used for ranking.

If a large learning platform doesn't show up in the results from the "other - please explain" option, it is likely due to a combination of "out of sight, out of mind," and the surveys not reaching an entire section of the industry.

SachaG commented 1 year ago

@mor10 thanks for the data! I personally don't have the time to get in touch with every platform on the list and ask them for their numbers, but if a third party like maybe https://www.classcentral.com/ could do that and establish some kind of objective ranking, I'd be happy to use that instead.

As for the surveys not reaching an entire section of the industry, that's definitely an issue but I'm afraid it doesn't have any easy solution…

mor10 commented 1 year ago

Just picking a random item on your primary list: what are the viewer and user numbers for Platzi? Are they more than 200,000 for a single course? If not, do you see how omitting a large platform because it also covers other focus areas is sub-optimal?

mor10 commented 1 year ago

Solving this issue likely means reformatting how you're grouping the options. Maybe split them into platform documentation (men, etc), blogs (CSS tricks), video learning (Udemy), other and have several questions. As long as you have a limited list you won't get reliable data. Whomever is on the primary list will always get the bulk of the votes, and if the primary list excludes entire platforms, your results are invalid.

SachaG commented 1 year ago

I think splitting up the items into more questions kinda ends up being the same as showing all items in fewer questions… I don't have a perfect solution to all this, but if we can already iterate towards something better than what we had before, I think that's a win.