search

HTTPArchive / almanac.httparchive.org

HTTP Archive's annual "State of the Web" report made by the web community
https://almanac.httparchive.org
Apache License 2.0
611 stars 170 forks source link

CSS 2020 #898

Closed foxdavidj closed 3 years ago

foxdavidj commented 4 years ago

Part I Chapter 1: CSS

Content team

Authors Reviewers Analysts Draft Queries Results
@LeaVerou @svgeesus @rachelandrew @fantasai @mirisuzanne @j9t @catalinred @hankchizljaw @dooman87 @rviscomi @leaverou Doc *.sql Sheet

Content team lead: @LeaVerou

Welcome chapter contributors! You'll be using this issue throughout the chapter lifecycle to coordinate on the content planning, analysis, and writing stages.

The content team is made up of the following contributors:

New contributors: If you're interested in joining the content team for this chapter, just leave a comment below and the content team lead will loop you in.

Note: To ensure that you get notifications when tagged, you must be "watching" this repository.

Milestones

0. Form the content team

1. Plan content

2. Gather data

3. Validate results

4. Draft content

5. Publication

rachelandrew commented 4 years ago

I'd be happy to help here. Layout is my CSS specialism but I can write about most things. I'm also an experienced editor.

j9t commented 4 years ago

I’m happy to contribute, too. With my current workload the least I can commit to is reviewing—I look forward to coordinating with whoever would be driving this section.

catalinred commented 4 years ago

I'd love to help in any way.

estelle commented 4 years ago

I have a particular knack and interest in selectors and specificity, but can write about other topics as well.

foxdavidj commented 4 years ago

@LeaVerou thank you for agreeing to be the lead author for the CSS chapter! As the lead, you'll be responsible for driving the content planning and writing phases in collaboration with your content team, which will consist of yourself as lead, any coauthors you choose as needed, peer reviewers, and data analysts.

The immediate next steps for this chapter are:

  1. Establish the rest of your content team. Several other people were interested or nominated (see below), so that's a great place to start. The larger the scope of the chapter, the more people you'll want to have on board.
  2. Start sketching out ideas in your draft doc.
  3. Catch up on last year's chapter and the project methodology to get a sense for what's possible.

There's a ton of info in the top comment, so check that out and feel free to ping myself or @rviscomi with any questions!

@rachelandrew @j9t @catalinred @estelle @hankchizljaw we'd still love to have you contribute as a peer reviewer or coauthor as needed. Let us know if you're still interested!

Andy-set-studio commented 4 years ago

Yeh just let me know when and if you need me 🙂

rviscomi commented 4 years ago

Tentatively adding @rachelandrew and @estelle as coauthors and everyone who commented as a reviewer. @LeaVerou feel free to reassign at your discretion.

LeaVerou commented 4 years ago

Howdy everyone! 👋

Warm thanks @rviscomi and @obto for selecting me to lead this effort. I'm psyched!

I think @rachelandrew would be an excellent co-author, especially for anything layout. @estelle, would you like to take on a selectors & specificity section? @svgeesus will be a co-author for Web Fonts & Colors

I have a lot of ideas for reviewers and co-authors, to which I will reach out privately, then @-mention here (I would rather not ask them in public, as it makes it harder to say no). I do not have any ideas for analysts however. Should we be looking for people swapping areas of responsibility or look for new people to join? Perhaps anyone from the current reviewers would like to become an analyst?

I'm going to re-read last year's chapter and reflect on what has happened in CSS within the year. A few quick initial thoughts:

If anyone has any ideas/suggestions, they are always welcome!

j9t commented 4 years ago

This is a world-class author team. I’m excited about the CSS chapter!

@obto, @LeaVerou, only to confirm, I’d absolutely be around for reviewing. And if the chapter is to dive deeper into the craft of writing CSS, I’d be happy and available to contribute more (it’s one of my focus areas).

catalinred commented 4 years ago

Hi everyone,

Happy that @LeaVerou accepted to be the lead author for the CSS chapter. The authors team is amazing.

Here are some of my random thoughts on what numbers I'd like to see in this new chapter:

Let me know what you think.

LeaVerou commented 4 years ago

I'd like to announce @fantasai as a reviewer.

@catalinred What fantastic ideas, thank you!

tunetheweb commented 4 years ago

What a fantastic line-up! Very excited to see what we come up with in this chapter.

LeaVerou commented 4 years ago

Announcing @mirisuzanne as reviewer!

j9t commented 4 years ago

Hi @LeaVerou, CSS chapter team—if I’m not missing anything then ID and class names hadn’t been covered in neither Markup nor CSS chapter last year. They may be more of a topic to cover in the Markup chapter but I wanted to raise this here in case you’ve flagged this on your side, too. Let’s coordinate on this if necessary—I’ll follow up as soon as I get a feeling for interest and feasibility on the Markup chapter side.

rviscomi commented 4 years ago

@estelle I've sent you an invite to join the 2020 Authors team on GitHub. Can you visit https://github.com/HTTPArchive/ to accept the invite? This will ensure you get notifications about important chapter milestones.

svgeesus commented 4 years ago

Variable fonts! Lots of interesting stuff to calculate there and I'm sure @svgeesus would love to.

I certainly would!

I'm unsure of the apportioning between this CSS chapter and the Fonts chapter though.

Popular vendor prefix numbers. Are the -webkit-, -moz-, -o- or -ms- numbers still relevant, considering the improved browser support for most of the CSS properties that once required vendor prefixes?

Nice one, although -webkit now falls into two categories which should be distinguished;

  1. Things that are actually WebKit specific
  2. Things that the other browsers have (wth varying degrees of reluctance) had to implement, primarily so that iOS-specific mobile content also works on browsers other than Safari. The -webkit vendor prefix merely being historical legacy, at this point.

The second one is interesting to analyze because it constitutes a part of the Open Web Platform that was added without the usual design debate or standards process.

svgeesus commented 4 years ago

@catalinred some great suggestions there!

svgeesus commented 4 years ago

I'm aware only Safari implements this now, but other browsers are showing interest so probing color(display-p3 r g b) would be timely this year. Interesting to see

LeaVerou commented 4 years ago

Alright people, some updates.

I think a single issue to manage which stats we are going to study (proposals, voting, methodology, discussion on the finer details of how to study them etc), would be a bit messy, so I made this repo. We should still keep this issue updated with resolutions etc, so that future authors can follow it.

I also made this app for one-click voting on issues that makes it easier to add the 👍 reactions. It's easy to change which repo its about, in case other chapter authors want to use it too.

Please post your stat suggestions as new issues in that repo and tag them proposed stat. Also please vote on existing stats (maybe give it a couple days for more stat proposals to be posted so they all get a fair chance). I already transferred all proposals I saw here and invited all co-authors and reviewers as collaborators in the repo, let me know if I missed anything!

I plan to tweet the link in a few days, so we get wider input from the community, but I'd like to solicit proposals and votes from contributors first.

foxdavidj commented 4 years ago

Hey @LeaVerou, just wanted to check in:

  1. Looks like things are moving along pretty smoothly. Is there anything you need from me to keep things moving forward, and have the chapter outline and metrics settled on by the end of the week?
  2. Can you remind your team to properly add and credit themselves in your chapter's Google Doc?
LeaVerou commented 4 years ago

@obto Well, we still don't have any analysts, so if you could help with that, that would be lovely!

Team (@LeaVerou @svgeesus @rachelandrew @estelle @fantasai @mirisuzanne @j9t @catalinred @hankchizljaw), please credit yourselves in the draft doc and request access if you don't have it!

LeaVerou commented 4 years ago

Hey team @svgeesus @rachelandrew @estelle @fantasai @mirisuzanne @j9t @catalinred @hankchizljaw,

Since we need to have decided on which stats we are studying by the end of this week, please post your suggestions as new issues in the repo: https://github.com/LeaVerou/css-almanac/issues as soon as possible so we have a few days for voting and wider community feedback.

foxdavidj commented 4 years ago

@LeaVerou Lot of awesome discussion going on here :tada: , but the Chapter doc could use some love. Think your team will be able to get an outline settled on by the end of the week?

LeaVerou commented 4 years ago

@obto I will try, though it's looking a little tight.

I've just posted on Twitter to solicit wider community feedback, so if anyone here is going to post more proposals, now would be the time to get them in, so they can get some votes.

svgeesus commented 4 years ago

@rachelandrew did you have any topics to add?

foxdavidj commented 4 years ago

@LeaVerou No worries. The main reason we have this week set as the deadline is because sometimes we come across metrics that require custom code (custom metrics) to be written and added to the Web Crawler -- which starts running on August 1st.

So if getting the chapter outline looks tough to complete... putting together a list of new metrics we didnt look at last year, that you're considering looking at this year, would be a big help.

foxdavidj commented 4 years ago

Fortunately @rviscomi implemented a script to parse all sites' CSS last year. Which should allow this chapter to do query almost everything already :)

LeaVerou commented 4 years ago

@obto I have posted the list a couple comments above :) You can find it here: https://github.com/LeaVerou/css-almanac/issues

I can write a list of those that may require custom code later today, but I need to know the details of this parser that @rviscomi used. How does it handle invalid properties/pseudo-classes/values? Does it still parse them, or is it equivalent to using document.styleSheets and friends in a browser? If it's the latter, which browser is it like? Also, what happens with JS? Some of the metrics we're considering would require looking into JS and correlating it with the CSS (Houdini, custom properties).

rviscomi commented 4 years ago

@LeaVerou the parser is https://github.com/reworkcss/css. It should parse everything, including invalid values. For example, see some of the fun invalid font-weight values explored in this post: https://discuss.httparchive.org/t/what-are-the-most-and-least-popular-numeric-font-weight-values/1732

rviscomi commented 4 years ago

Adding myself as an analyst to help this chapter out.

@HTTPArchive/analysts is anyone else available to share the analysis for this chapter?

dooman87 commented 4 years ago

@rviscomi I'd like to help out. This is my first year, so I would need a bit of guidance. I got BigQuery part, but still learning what other things analysts do :)

rviscomi commented 4 years ago

Great thanks @dooman87! Can you request access to the doc so we can help triage metrics as they get planned?

LeaVerou commented 4 years ago

It just occurred to me that we could get HUGE insight about what CSS authors need by analyzing Sass stylesheets! We can find URLs to Sass files from the CSS files that have sourcemaps and there is a JS-based Sass parser. This will give us insight not just into the current state of CSS, but perhaps more importantly, its future.

@HTTPArchive/analysts @obto @rviscomi is this possible?

In other news, I've been looking into Rework CSS to see what kind of information its AST gives us. For anyone else who wants to experiment, I've made an online playground here. You can also use it as a template. It appears like it's doing rather loose parsing, which is good on one hand because it does not need to be constantly updated for new syntax, but only for major grammar changes, so it parses a very wide range of CSS. On the other hand, it means a lot of analysis would still need to be done via regexes on the JSON.

I would also be willing to help out with the analysis, but I'm not sure yet if I can commit properly to being one of the analysts, I'm reading the guide etc and will know soon.

rviscomi commented 4 years ago

It just occurred to me that we could get HUGE insight about what CSS authors need by analyzing Sass stylesheets!

We're limited to whatever resources are served over the network during the page load. IIUC sourcemaps only point to the source stylesheets, but they're not actually served with the page, so we wouldn't have the sources in the dataset :(

I would also be willing to help out with the analysis, but I'm not sure yet if I can commit properly to being one of the analysts, I'm reading the guide etc and will know soon.

Great, the team would be happy to have you! Let @paulcalvano or me know when you decide and if you need any help getting started.

In other news, I've been looking into Rework CSS to see what kind of information its AST gives us. For anyone else who wants to experiment, I've made an online playground here.

The Rework playground is a great idea. Could you also test it against the version used in the 2019 queries? I modified it a bit for efficiency, so I plan to reuse that unless there are any major changes we're missing.

LeaVerou commented 4 years ago

We're limited to whatever resources are served over the network during the page load. IIUC sourcemaps only point to the source stylesheets, but they're not actually served with the page, so we wouldn't have the sources in the dataset :(

I reached out to @obto over email, as I was keen to get a response asap, due to the tight timeline, and he said it may be possible:

The difficulties in pulling something like that off, is the Crawler only saves the response bodies of the URLs that the browser fetches. So we'd need to write a script to run after the page loads and fetch it, which is something the SEO chapter was talking about since they want to analyze the robots.txt file. I'm not sure how that proposal has been progressing but I'll check it out and get back to you

Is there a document detailing how the crawler works? E.g. if it can run dev tools, then sourcemaps would be fetched.

The Rework playground is a great idea. Could you also test it against the version used in the 2019 queries? I modified it a bit for efficiency, so I plan to reuse that unless there are any major changes we're missing.

Sure, here you go: https://codepen.io/leaverou/pen/PoZVeeE And to use as a template: https://codepen.io/pen/?template=PoZVeeE I can check if there are any major changes, do you know which commit yours was forked from?

rviscomi commented 4 years ago

Is there a document detailing how the crawler works? E.g. if it can run dev tools, then sourcemaps would be fetched.

https://almanac.httparchive.org/en/2019/methodology#tools is the best doc we have on the crawl tech. At its core, it's powered by WebPageTest. As of now I haven't found a reliable way to instrument it for HTTP Archive to load arbitrary resources in a way that doesn't interfere with the site's loading performance. cc @pmeenan

I can check if there are any major changes, do you know which commit yours was forked from?

Thanks! Looking at https://github.com/reworkcss/css/tags it seems like 3.0 was released a few weeks ago and the most recent release prior to that was 2.2.4 in 2018. Also parse/index.js doesn't actually seem to have been changed since 2015?? So maybe nothing changed?

LeaVerou commented 4 years ago

Alright, an initial chapter outline has been updated in the doc and on Github, though we are still iterating. I did include Sass for now, since feasibility is evaluated later, and hope dies last. 😁

LeaVerou commented 4 years ago

I would also be willing to help out with the analysis, but I'm not sure yet if I can commit properly to being one of the analysts, I'm reading the guide etc and will know soon.

After a lot of back and forth with myself (I'm very interested, but worried about committing to doing yet another thing), I think I want to try, if it's not too late. What's the process to add myself @rviscomi?

rviscomi commented 4 years ago

Not too late at all! This chapter is shaping up to be very data-heavy (in the best possible way) so having more analysts on the team will be a big help. The process is:

  1. Edit https://github.com/HTTPArchive/almanac.httparchive.org/issues/898#issue-646592132 to add yourself to the Analysts column
  2. Submit a PR to add yourself to the analysts team in src/config/2020.json
  3. Review the Analysts' Guide and let me or @paulcalvano know if you have any questions
LeaVerou commented 4 years ago

@rviscomi Done!

foxdavidj commented 4 years ago

@LeaVerou @dooman87 @rviscomi for the two milestones overdue on July 27 could you check the boxes if:

Keeping the milestone checklist up to date helps us to see at a glance how all of the chapters are progressing. Thanks for helping us to stay on schedule!

rviscomi commented 4 years ago

I've updated the chapter metadata at the top of this issue to link to the public spreadsheet that will be used for this chapter's query results. The sheet serves 3 purposes:

  1. Enable authors/reviewers to analyze the results for each metric without running the queries themselves
  2. Generate data visualizations to be embedded in the chapter
  3. Serve as a public audit trail of this chapter's data collection/analysis, linked from the chapter footer
LeaVerou commented 4 years ago

Hey editors & reviewers (@svgeesus @rachelandrew @estelle @fantasai @mirisuzanne @j9t @catalinred @hankchizljaw)! Since the bulk of the analysis will be JS on the Rework AST, @rviscomi and I were wondering if any of you would be willing to help with the JS? We have over 40 stats to calculate and only 2.5 analysts.

svgeesus commented 4 years ago

I came across a comment in an old CSS Color 4 issue where I wondered aloud whether there was Web-compat risk on out of range sRGB values. I suspect not, but it would be nice to have data on whether sRGB values outside the range [0-255] or [0%-100%] are in actual use.

In practice browsers will either do per-component clip on these or treat them as invalid. The spec was only changed to require per-component clipping in 2016.

If it is easy to look for those out of range values in the wild,it would be a nice to have.

LeaVerou commented 4 years ago

@rviscomi Just spotted a bug with our Rework parser: It seems to parse @import statements incorrectly when the URL contains a colon. E.g. try this:

@import url('https://fonts.googleapis.com/css2?family=Playfair+Display:ital,wght@0,400..900;1,400..900&display=swap');
@import url('https://fonts.googleapis.com/css2?family=Public+Sans:wght@100..900&display=swap');

foo {
}

I get:

{
    "type": "stylesheet",
    "stylesheet": {
        "rules": [
            {
                "type": "import",
                "import": "url('https://fonts.googleapis.com/css2?family=Playfair+Display:ital,wght@0,400..900"
            },
            {
                "type": "rule",
                "selectors": [
                    "1,400..900&display=swap'); @import url('https://fonts.googleapis.com/css2?family=Public+Sans:wght@100..900&display=swap'); foo"
                ],
                "declarations": []
            }
        ],
        "parsingErrors": []
    }
}
rviscomi commented 4 years ago

Good catch! If you know of a fix, please patch css-parser.js. We can regenerate the parsed_css table if needed but I'd imagine the impact is small.

svgeesus commented 4 years ago

Since the bulk of the analysis will be JS on the Rework AST, @rviscomi and I were wondering if any of you would be willing to help with the JS?

I just sent a PR for JS to see if P3 colors out outside the sRGB gamut

Tiggerito commented 4 years ago

Hi CSS team. Tony here from the SEO team.

I've had a request to create a report in css media usage. Basically as a way to see if a design is responsive or not. Any chance on a pointer in the right direction on how to pull out that info?

Cheers.

LeaVerou commented 4 years ago

@Tiggerito I suppose your best bet is to check if there are any dimension media queries (with width, height, or aspect ratio). The JS here may be a good starting point (you could use the same JS and just filter the list to the features you're interested in), however there is no SQL for that yet. @rviscomi perhaps we could prioritize the SQL for that query so that @Tiggerito can base his off that?

rviscomi commented 4 years ago

Sure, I'm behind on queries for this chapter but I can prioritize this one first. Lea, I just noticed this JS also includes the incompatible ?. syntax that we've had to remove from other queries. Would you be able to rewrite this one slightly to make it work in BigQuery? It would be great to also do a sweep of the JS to make sure it's not hiding anywhere else.