search

Devographics / StateOfCSS-2020

The State of CSS 2020 survey results
https://2020.stateofcss.com
Other
45 stars 14 forks source link

State of CSS 2021 Questions Feedback Thread #8

Open SachaG opened 3 years ago

SachaG commented 3 years ago

(Updated August 17, 2021)

We are getting ready to hold the 2021 edition of the annual State of CSS survey (see the 2020 edition here).

Please use this issue thread to give us your feedback on the survey outline used to generate the survey questions:

Process

Generating the outline is a three-part process:

  1. First, we collect as many inputs, suggestions, and ideas as we can.
  2. Then, we evaluate all the item we've received and try to narrow them down to a reasonable number of options.
  3. Finally, we post the reworked outline here for a last round of feedback.

Timeline

  1. August 17: outline open to feedback
  2. September 1: deadline for adding new items
  3. September 5: outline finalized

Useful Links & Resources

SachaG commented 3 years ago
web-padawan commented 3 years ago
fantasai commented 3 years ago

+1 to @Kilian’s 2020 suggestion to mention scroll-margin. I'd pair it with scroll-padding, which actually should work better for the fixed-header case. They work even if you don't have snapping turned on.

We very strongly encourage devs to not use direction, so I wouldn't list it. Probably fewer people know it exists the better... But you could ask about :dir()!

On media queries, I would suggest the pointer and hover queries, as they're very useful for switching layouts for tablet vs. desktop, but I suspect most people don't know about it. I think prefers-contrast and forced-colors will be two other interesting ones, but they're fairly bleeding-edge and not widely implemented, but maybe a good idea for next year.

+1 to @argyleink’s suggestion of min-content/max-content

I would add text-decoration-style/text-decoration-color and break-inside/break-before/break-after.

I wouldn't include exclusions. it's not implemented except in old IE engines and doesn't have much forward momentum.

KittyGiraudel commented 3 years ago

Hello there,

First of all, I would like to apologise to Sacha for not having had the time to check the State of CSS before it was released. I appreciate you contacted me to have my opinion, and I should have invested more effort in finding the time to go through it. I have some constructive comments that I think would have been benefitial. I hope they can be taken into consideration for 2021.

I’d like to focus a little on the “Resources” stage of the survey, since it has spun some discomfort on Twitter, and I also felt a little awkward going through it myself. Let’s start by talking about the “Blogs & Magazines” section. I think that makes sense to list the big names in the industry, such as CSS-Tricks, ALA, Smashing Magazine, Codrops, SitePoint, Dev.to and the like.

That being said, if we’re going to list eponym blogs (which I assume is the case since David Walsh’s blog is mentioned), it would have been nice to include more people. Now whether listing dozens and dozens of blogs is a good idea is a different story, but listing David Walsh’s blog as the only person’s name in that list is a bit of an odd move in my opinion—nothing against David himself, he’s a lovely person and I got the opportunity to write guest posts for his site. The same comment can be made for the unique presence of Wes Bos in the “Sites & Courses” section. There again, no offence intended whatsoever towards Wes Bos; he’s a wonderful teacher and a great content producer. My point is that surely there are other people who deserve to be listed in there just as much.

Now, I appreciate there are free text fields for people to input the sites, blogs and courses they regularly follow, but that’s a little besides the point in my view. For all intents and purposes, this survey appears as an authority about CSS. It lists the industry standards in every section: which frameworks and preprocessors are worth mentioning, which CSS-in-JS libraries and methodologies are worth considering… Therefore one can wonder whether it should mention individuals at all, and if it does, it might be good to take this opportunity to acclaim people whose work is often forgotten.

I understand this is a difficult topic, and there is no one right answer or one way to do things. Here are my personal suggestions—do what you will with this information:

  1. If the goal of the “Resources” stage is to collect resources used by frontend developers, I think relying solely on free text might be a better approach. This way, people are not led towards a specific answer, and can mention which sites they effectively use on a regular basis. Additionally, the survey doesn’t imply some resources are worth checking more than others which is nice.

  2. Mentioning people by name feels a little like a leading question and probably should be avoided in my opinion. When I read some people’s name in the current state of the list, I can acknowledge they are talented individuals who made a difference for many developers while also wondering why so many people who shaped the field are not mentioned. It feels a little awkward to me.

  3. It could be interesting to use the reach of this survey to effectively show people other resources—particularly from people who are usually not mentioned or rewarded. If we do not want to influence the results of the survey (which is understandable) then maybe this could be done once it’s submitted. It could be something like “By the way, have you heard of these people & sites…?”, the same way it currently mentions CSS properties to learn more about. It could even be populated by previous answers (see my point #1).

I guess my TL;DR is that I think the State of CSS should either make an honest and good effort at mentioning, listing and acclaiming the work of the many individuals who shaped the CSS field (for lack of a better term) and who continuously deliver quality content, or should refrain from mentioning individuals at all in order to appear less biased (although I can totally appreciate this was done without any ill intent of any kind).

Note that this comment in no way takes away the effort and quality of the work that went into this survey. It’s comprehensive, well done, and insightful as always. I do appreciate the initiative, and hope you don’t mind sharing my opinion. Thank you for your time and your work. 💖

SachaG commented 3 years ago

@HugoGiraudel that's great feedback! To address your points:

  1. Free text fields are a bit problematic because they're harder to parse on our end, so you need more work for less reliable data.
  2. I agree, originally I wanted to only include non-personal blogs but made an exception for David Walsh's blog as he publishes posts by other authors.
  3. That's also an interesting option, but at the same time we want to avoid being the gatekeepers of who is or isn't mentioned in the survey even more than we currently are… I feel like picking and choosing who we highlight would be putting our thumb on our scale a lot.

Currently the process for picking resources, frameworks, etc. is that we come up with the best list we can the first year we do a survey, then look at the "others" results to augment the list for the following year, and so on. It's not a perfect process, and in fact I may have missed some of the options for this year's CSS survey (it's hard to think of everything…), so that's definitely on me.

I think moving forward the best thing to do would be to find another, different way to highlight the important people in the CSS community. For example, I was thinking we could ask people to link their Twitter account so we could cross-reference their follow lists and who the most followed developers are (of course assuming we find a way to filter just CSS-related people, which I'm not sure is possible…).

Or maybe we could come up with our own list of people and see whether respondents follow them or not – but that would require first establishing an impartial criteria for selecting such "important CSS people".

SachaG commented 3 years ago

I also did want to mention that we also don't want to fall into the opposite extreme of rejecting people or resources because they're not "CSS-y" enough. Front-end developers exist on a very diverse spectrum (JS, CSS, both, etc.) and we definitely wouldn't want to be in the position of excluding certain people because they don't fit the "correct" profile of a CSS developer, or aren't part of the right crowd.

eclecticjohny commented 3 years ago

Just wondering where you get the list of podcasts for the survey as some seem to be outdated and not active? Is there a chance for some better choices of more active podcasts?

SachaG commented 3 years ago

@eclecticjohny we get it from right here! Which ones do you think we should add/remove?

eclecticjohny commented 3 years ago

The Web Ahead and Non Breaking Space are both examples of non-updated podcasts. The CSS Podcast or Frontend First are examples of ones I can think of that are actively produced.

mariasimo commented 3 years ago

Hi! Maybe a little off-topic but I was reviewing results of the survey this year (big fan, btw :) ) and I am missing having my results from when a fill it in a mail or something, just to compare, learn the lessons to be learned, etc. I think it would be great to have a way to read own results. Maybe already exists and I missed it? Anyway, thanks for the work, super cool initiative!

SachaG commented 3 years ago

@mariasimo we don't send out an email but you can go back to http://survey.stateofjs.com/ and check out what you filled in anytime.

mariasimo commented 3 years ago

@mariasimo we don't send out an email but you can go back to http://survey.stateofjs.com/ and check out what you filled in anytime.

Ok, already saw it! Thanks 🤓

SachaG commented 3 years ago

Remove Radium (see https://github.com/StateOfJS/StateOfCSS-2020/pull/29)

SachaG commented 3 years ago

Add intrinsic sizing: https://ishadeed.com/article/intrinsic-sizing-in-css/

SachaG commented 3 years ago

@property https://css-tricks.com/exploring-property-and-its-animating-powers

SachaG commented 3 years ago

Btw regarding the feedback that last year's "blogs and sites" section wasn't very representative of the CSS community I think I'm just going to add a more inclusive "people" section this year.

It'll be a lot of names, but I was looking at the great blogroll at the bottom of Hidde De Vries' blog and seeing all these familiar names (and discovering a few) is actually pretty cool so I don't think people will mind.

una commented 3 years ago

Just some ideas/suggestions:

foolip commented 3 years ago

For the "what's missing from CSS" question, since we know from last year roughly what the areas are, can we make it a "pick 3 plus write in if other" type of question?

tomayac commented 3 years ago

Color theming is another interesting growing area (i.e. light and dark theme). Could we ask if folks provide a light, dark, or multiple themes?

Maybe also hinting at the color-scheme property, which is less well-known but pairs well with the popular prefers-color-scheme user preference media feature.

foolip commented 3 years ago

Would it be possible to ask about interoperability in this survey? I.e. an option for "would use a property if it was supported cross browser"

I love this suggestion, and getting some signal about where web developers are most struggling with current or emerging CSS technologies is one of the main reasons I've wanted to get involved in State of CSS. I think it's a good idea to ask specifically about browser support, before I had envisioned a more generic "struggling with" question but that then conflates things that are inherently difficult to get right (but works everywhere) with things like flex gap which people couldn't use because browser support wasn't there.

Letting developers pick between all css properties would be an option, but that notably won't work for things like flex gap where the property is gap but the problem was specifically in flex layout. Free form responses would work, and I'd be happy to help analyze some of the responses, but is there another way?

argyleink commented 3 years ago

a few to consider:

bfgeek commented 3 years ago

One thing I'd be interested in getting signal on is there any bad performance bugs/problems developers are running into. (I don't think State of CSS has covered this area before, being more feature focused? - Happy to be wrong!)

For example: Are people struggling creating smooth animations? Have people run into issues where the browser seemed to take too long to visually update? Janky scroll prevalent? Input lag?

Unsure how to ask this question, as don't particularly have a hypothesis to test, a free form answer might get some signal. E.g. something like: "In the past 12 months what types of browser performance issues have you run into?"

SachaG commented 3 years ago

@una

  • I agree that it would be good to ask about scroll APIs, maybe also @scroll-timeline

We already had scroll-snap, do we also need scroll-padding/scroll-margin as separate items? Or is that redundant? I'm worried about the survey getting a bit too long…

Color theming is another interesting growing area (i.e. light and dark theme). Could we ask if folks provide a light, dark, or multiple themes?

We ask about dark themes as part of the "Accessibility Features" question. Maybe we can reword it as a more generic "color themes" instead.

Would it be possible to ask about interoperability in this survey? I.e. an option for "would use a property if it was supported cross browser"

Maybe, if you have a suggestion on how to phrase it? Maybe it could be an opinion question? (e.g. "Browser interoperability is a big ongoing concern for me"). Personally it's a concern in the abstract but it rarely stops me from using a property I want in practice since I assume other browsers are going to catch up eventually, I don't know if we can find a way to phrase that question that captures these nuances.

@foolip

For the "what's missing from CSS" question, since we know from last year roughly what the areas are, can we make it a "pick 3 plus write in if other" type of question?

Great idea. Would this list work? To be honest the big two are really container queries and parent selector, after that it's a lot more fragmented. 

        options: 
          - id: container_queries
          - id: parent_selector
          - id: nesting
          - id: functions
          - id: browser_support
          - id: mixins
          - id: scoping

@argyleink

accent-color and ::marker seems like good ones maybe? I haven't really heard about the others. To be honest it's a little bit tough to decide what to feature or not, it's easy enough to find esoteric CSS properties that people don't know about, but if they're too niche then including them in the survey doesn't make a ton of sense… The idea is more to find upcoming properties that will make a big impact soon, like grid a few years ago for example.

@bfgeek

It sounds like we could have a "what areas of CSS do you struggle with the most" question? Happy to take more suggestions for options.

Also as general feedback, we try to stay away from open-ended, qualitative questions ("what performance issues do you run into?") because we can't really do much with the resulting data. Also we want the survey to remain easy to take, asking people to stop and think kinda slows down their momentum.

Anyway thanks a ton for all the great suggestions, I appreciate you all taking the time to review this!

foolip commented 3 years ago

Also as general feedback, we try to stay away from open-ended, qualitative questions ("what performance issues do you run into?") because we can't really do much with the resulting data. Also we want the survey to remain easy to take, asking people to stop and think kinda slows down their momentum.

Open ended questions are definitely a last resort and take more time for both survey takers and to make sense of the results, but it can still be really useful. What do you feel is currently missing from CSS? was used by the Google Chrome team at least. That being said, one of the great things with State of CSS is it's pretty short and people seem to enjoy taking it, so it should be a high bar to add time-consuming questions, including the ones I'm most excited about :)

SachaG commented 3 years ago

Oh for sure, they can be useful but we need to do a lot of normalization work behind the scenes to pattern-match the most common answers, so it's time-consuming to add new ones…

As for the "what areas of CSS do you struggle with the most with?" question, how about those 8 options as a starting point?

  1. Browser interoperability
  2. Keeping up to date with new properties
  3. Code architecture/methodology
  4. Maintaining codebases over time
  5. Performance issues
  6. Positioning elements on screen
  7. Remembering the syntax
  8. CSS Specificity issues

With 8 items we can use this new "tournament bracket" question type I've been working on to help people rank the items:

Screen Shot 2021-09-01 at 11 17 50
foolip commented 3 years ago

For the "what's missing from CSS" question, since we know from last year roughly what the areas are, can we make it a "pick 3 plus write in if other" type of question?

Great idea. Would this list work? To be honest the big two are really container queries and parent selector, after that it's a lot more fragmented.

        options: 
          - id: container_queries
          - id: parent_selector
          - id: nesting
          - id: functions
          - id: browser_support
          - id: mixins
          - id: scoping

@una can you help review this? I suspect that at least "functions" needs to be split into color functions and potentially other functions. If we ask about interop/compat separately, perhaps we can drop "browser support" here.

foolip commented 3 years ago

@SachaG for "what areas of CSS do you struggle with the most with?" we have some precedent for this in Layout/styling feature that cause issues from the MDN browser compat survey: image

(Ignore the typo...)

It's very clear from many surveys that browser compat is a top pain point. Just repeating the breakdown from that question probably won't yield anything new, but I think we do have to get to concrete areas, not just browser compat/interop as a whole. This is ultimately to guide prioritization for a future iteration of Compat 2021.

The question I have in mind is "please name a few features that you can't use like you want because of browser compat/bugs" in some other words. For responses, I think either an autocompleting list of all CSS properties/functions/etc. or simply free form text.

SachaG commented 3 years ago

Btw there's also some general feedback about CSS issues here: https://twitter.com/SachaGreif/status/1432894080799821828

SachaG commented 3 years ago

OK, here's the final list of 8 "pain points" I came up with based on the MDN survey:

  1. Browser Interoperability
  2. Animations
  3. Architecture & Maintenance
  4. Layout & Positioning
  5. Scoping & Specificity
  6. Responsive Design
  7. Form Elements Styling
  8. Performance

The question I have in mind is "please name a few features that you can't use like you want because of browser compat/bugs" in some other words. For responses, I think either an autocompleting list of all CSS properties/functions/etc. or simply free form text.

OK, I get the idea. I don't think it's a question I would've included myself, since I would worry it wouldn't apply to most people, who I assume gravitate towards more established features anyway, and worry less about incompatibilities. But maybe I'm wrong.

But anyway if it's going to be helpful to you we can include it for sure. At this stage I don't have a great idea of how to do this apart from a free form text field so let's go with that. Maybe just "Are there any CSS features that you haven't been able to use because of browser compatibility issues?"

SachaG commented 3 years ago

Oh also a question to you all, but what would you think of leaving out the "Units & Selectors" section in order to keep the survey shorter? If there are new units and/or selectors we can just merge them into the main features section.

SachaG commented 3 years ago

I've updated https://survey.stateofjs.com/survey/state-of-css/2021/ btw.

foolip commented 3 years ago

OK, here's the final list of 8 "pain points" I came up with based on the MDN survey:

  1. Browser Interoperability
  2. Animations
  3. Architecture & Maintenance
  4. Layout & Positioning
  5. Scoping & Specificity
  6. Responsive Design
  7. Form Elements Styling
  8. Performance

I think that's a pretty good list in that it captures most of what people would probably think about as pain points. But I think that with only a fixed number of high-level-ish options, we won't be able to learn anything new. Or at least I can't think of a result to the question which would be surprising and actionable.

The question I have in mind is "please name a few features that you can't use like you want because of browser compat/bugs" in some other words. For responses, I think either an autocompleting list of all CSS properties/functions/etc. or simply free form text.

OK, I get the idea. I don't think it's a question I would've included myself, since I would worry it wouldn't apply to most people, who I assume gravitate towards more established features anyway, and worry less about incompatibilities. But maybe I'm wrong.

I think there is an effect where features that aren't broadly supported don't get used much, and so developers might not learn about them and not see it as a problem. But it's still the case that browser compatibility in some form or another bubbles to the top of pain points or wants across surveys, and when asked to give specific examples we've heard things like subgrid or flex gap, which at the time weren't broadly supported. (Subgrid still isn't, but flex gap is.)

But anyway if it's going to be helpful to you we can include it for sure. At this stage I don't have a great idea of how to do this apart from a free form text field so let's go with that. Maybe just "Are there any CSS features that you haven't been able to use because of browser compatibility issues?"

That's a great suggestion, and that would be very helpful, thank you! I wonder if there's a tweak to "haven't been able to use" that could also invite people to think about cases where they've had to work around some issue but could use the property. I'm not sure how to phrase it, so leaving it as a suggestion for now.

SachaG commented 3 years ago

But I think that with only a fixed number of high-level-ish options, we won't be able to learn anything new.

Yes I would also add a freeform "other CSS pain points" question next to the ranking one.

I wonder if there's a tweak to "haven't been able to use" that could also invite people to think about cases where they've had to work around some issue but could use the property.

Maybe "struggled to use"? Or "encountered difficulties using"?

foolip commented 3 years ago

"Are there any CSS features that you've struggled to use because of browser compatibility issues?" with free-form text replies sounds great to me!

I would analyze this in a similar way as we did for the compat survey, extracting up to 5 things per answer. Hopefully most people will list only 1 or 2 things :)

tomayac commented 3 years ago

What recently came up in CSS Flex world was flex-end alignment and scrolling. See crbug/411624 for details and petite-smooth-honeydew.glitch.me for a reduced test case.

SachaG commented 3 years ago

Stuff I'm removing from the survey:

foolip commented 3 years ago

I have stepped through https://survey.stateofjs.com/survey/state-of-css/2021/ and have some more feedback.

In "What do you feel is currently missing from CSS?" and any other multiple-choice question where we're looking for an importance signal, it would be good to limit to 3 choices, to force a prioritization. Otherwise the risk is that most people select all the things they'd like, giving a very flat signal. (This is advice I got from researchers, at least.)

Also, I wonder if some of these options could use an explanation in a parenthical?

foolip commented 3 years ago

As an alternative to limited the choice to 3, if we could add two things to the list, it could also be a tournament style question.

argyleink commented 3 years ago

very exciting step through!

my only confusion moment was between Environments > Browsers and Other Tools > Browsers, felt like i was asked the same question back to back, but upon review, the difference was "Which browsers do you test in?" vs "Which browser(s) do you primarily work in during initial development?". no change request, just kinda documenting the moment i had.

i've got to shamelessly plug https://web.dev/learnCSS as a Resources > Sites & Courses candidate. But feel no pressure to add it, i just gotta !!say it.

SachaG commented 3 years ago

As an alternative to limited the choice to 3, if we could add two things to the list, it could also be a tournament style question.

Any suggestions?

Also, I just read an article about @layer, should we include that too in the survey?

argyleink commented 3 years ago

I could def see @layer being in the survey

SachaG commented 3 years ago

OK I'll add that but guys please stop coming up with new CSS features for at least the next 2 months!

foolip commented 3 years ago

As an alternative to limited the choice to 3, if we could add two things to the list, it could also be a tournament style question.

Any suggestions?

Looking at the 2020 results, "Functions" is the category that came up the most, and in https://whatsmissingfromcss.com/data/comments.json different kinds of color functions seem the most common type of function. So I would nominate "Color functions". @una do you think that makes sense?

The next runner up is Subgrid, so I would just add that. It's still only supported in Firefox, and a strong signal that developers want it would help prioritization.

That should be 8 things for "what's missing". I'm assuming here that @layer got added elsewhere.

una commented 3 years ago

I think it’s a bit early for @layer to be in the survey. - my two cents. Color functions makes more sense to include though!

On Fri, Sep 17, 2021 at 10:49 AM Philip Jägenstedt @.***> wrote:

As an alternative to limited the choice to 3, if we could add two things to the list, it could also be a tournament style question.

Any suggestions?

Looking at the 2020 results, "Functions" is the category that came up the most, and in https://whatsmissingfromcss.com/data/comments.json different kinds of color functions seem the most common type of function. So I would nominate "Color functions". @una https://github.com/una do you think that makes sense?

The next runner up is Subgrid, so I would just add that. It's still only supported in Firefox, and a strong signal that developers want it would help prioritization.

That should be 8 things for "what's missing". I'm assuming here that @layer got added elsewhere.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/StateOfJS/StateOfCSS-2020/issues/8#issuecomment-921859078, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM5L3DJYP4VM6HZTWOZ3NTUCNIRHANCNFSM4SVVFA2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

SachaG commented 3 years ago

OK, I made the missing features question into a bracket:

https://survey.stateofjs.com/survey/state-of-css/2021

Screen Shot 2021-09-19 at 9 03 42
foolip commented 3 years ago

I've tried the bracket questions again and they're quite fun to fill in. One bit of feedback about the "Start Over" mechanism. It looks like it randomizes the order of the options again, which doesn't seem great if someone just made a wrong choice and wants to quickly do it over again. They'd have to consider new pairs due to the randomization.

Second, I wonder if the "Start Over" button could be visible at all times? If one makes a mistake early on, one doesn't yet know it's possible to start over and might skip the question or bail from the survey.

SachaG commented 3 years ago

I agree on both counts but basically this is the best I could come up with for now…

foolip commented 2 years ago

How about always showing the "Start Over" button, would that not be easier than conditionally showing it?

SachaG commented 2 years ago

True, but I like how the fact that the "start over" button appears at the end currently lets you know that you're finished with the question. I think this is a case where I'll wait to see what feedback early survey takers have about the format.

foolip commented 2 years ago

Oh, right, it's not necessarily obvious when you're done without that state change. Awaiting early feedback sounds good!